Preheat your oven to 350°F (175°C). Generously butter one 9-inch round cake pan or springform pan.
In a food processor, pulse the whole almonds until they form a fine, flour-like consistency. Be careful not to over-process into butter.
In a medium bowl, whisk together the ground almonds, all-purpose flour, baking powder, and salt.
Set aside.
In a large bowl using an electric mixer, beat the 8 tablespoons (120 ml / 4.1 fl oz) of softened butter and the sugar on medium-high speed until light, creamy, and pale in color.
Add the eggs one at a time, beating well after each addition until the mixture is thick and fluffy, about 5 to 7 minutes. Scrape down the sides of the bowl as needed.
Beat in the vanilla or almond extract and the lemon zest.
On low speed, add the dry ingredient mixture to the wet , mixing until just combined. Gently mix in the milk until the batter is smooth.
Pour the batter into the prepared pan and spread it into an even layer.
Bake for 30 to 40 minutes, or until a toothpick inserted into the center comes out clean.
Let the cake cool in the pan on a wire rack for 15 minutes. Carefully invert the cake onto the rack to cool completely before glazing, frosting, or slicing.
Once you add the flour mixture, mix only until just combined. Overmixing develops gluten and can result in a tough cake.