Remove tough ends from asparagus stalks. Cut stalks into 1-inch pieces. Set aside.
In a large pot, heat oil. Saute leeks until tender. Add garlic and saute for 1 minute. Add asparagus stalks and chicken broth. Bring to a boil. Lower heat and partially cover pot. Simmer until vegetables are very tender, about 20 minutes.
Transfer soup to a high-speed blender and puree. Return soup to pot. Add milk and stir to combine. Add salt and pepper to taste. Serve warm.