In a large mixing bowl, cream together the butter, peanut butter, granulated sugar, and brown sugar until smooth and fluffy, about 3-5 minutes with an electric mixer. Add the egg and vanilla extract, mixing until fully incorporated.

Preheat your oven to 350°F (175°C) and line two baking sheets with parchment paper. Scoop tablespoon-sized portions of the dough, rolling them into balls, and place them on the baking sheets, leaving space in between. Flatten each ball slightly and sprinkle with granulated sugar. Bake for 10-12 minutes, until edges are light golden. Let cool for 5 minutes before transferring to a wire rack.

Prepare the creamy filling by beating together peanut butter, unsalted butter, powdered sugar, and vanilla extract in a mixing bowl until smooth. If too thick, gradually add milk to desired consistency.

Once cookies are cool, spread filling on the flat side of one cookie, place another cookie on top, and gently press together. Repeat until all cookies are filled and sandwiched.