You probably know you can access the clipboard on Windows by pressing Win + V. This opens up a pane with various tabs, including one for pasting emojis. However, a direct way to access the emoji pane (that I just discovered!) is by pressing Win + . (period key).
For AZERTY keyboards, use Win + ; (semicolon key) instead.