Preheat the oven to 350°F and grease an 8x8-inch square baking pan with cooking spray or line with parchment and set aside.
To prepare the streusel topping, mix together the gluten-free flour blend, brown sugar, and cinnamon in a medium bowl. Cut in the cold butter to the flour mixture, until the butter pieces are coated with flour and you have some pieces the size of peas. Set aside.
For the cinnamon swirl, mix together the brown sugar and cinnamon until combined. Set aside.
To make the cake, add the gluten-free flour blend, xanthan gum (if using), baking powder, baking soda, and salt to a large bowl. Whisk until combined.
Place the butter and granulated sugar in a separate mixing bowl. Use an electric mixer and beat until light and fluffy, about 2-3 minutes.
Add the eggs, sour cream, milk and vanilla. Mix until smooth. Scrape down the sides of the bowl with a spatula as needed.
Add the dry ingredients to the wet ingredients and mix until just combined and no more streaks of flour remain.
Spoon half of the batter into the prepared baking dish. Smooth with a spatula.
Sprinkle the cinnamon sugar mixture evenly over the batter.
Cover the cinnamon sugar with the remaining batter. Spread with a spatula, being careful not to disturb the sugar.
Sprinkle the gluten-free streusel over the batter.
Bake at 350F for 30-40 minutes, or until the cake is a light golden brown and when a toothpick is inserted into the center it comes out clean.
Let the coffee cake cool completely.
Prepare the glaze by adding the powdered sugar, 1 tablespoon of the milk, and vanilla extract to a small bowl. Whisk until combined. If the glaze is too thick add more milk until it reaches your desired consistency.
Drizzle the glaze over the coffee cake and allow to set before serving.