Preheat the oven to 350F. Grease a 9x13-inch baking pan or line with parchment paper. Set aside. In a large bowl, whisk together the gluten-free flour, xanthan gum (if using), baking powder, baking soda, salt, ground cinnamon, ground nutmeg, and ground ginger.
In a separate bowl, add the ripe bananas. Mash with a fork until thoroughly mashed and liquidy.
Add the sour cream, vegetable oil, granulated sugar, brown sugar, eggs and vanilla extract. Whisk until smooth.
Gradually add in the dry ingredients, mixing until no more streaks of flour remain.
Pour the banana cake batter into the prepared baking pan.
Bake at 350F for 45-50 minutes, until a toothpick inserted in the center comes out clean with a few moist crumbs. Cool completely before frosting with frosting.
Frost with brown butter cream cheese frosting (recipe below) and sprinkle with chopped walnuts.
Slice and serve.