Found the source of the bug after spending way too much time messing with settings. Fully reproducible. It is directly related to GSync-enabled high refresh rate monitors.
Steps to generate bug:
- Enable GSYNC on fullscreen mode (or both fullscreen and windowed mode as this will generate the stuttering as well)
- Enter game and perform action that generates a notification
- Open and close notification window, game will stutter for about 0.5 seconds.
- Continue playing and the stutters generally appear randomly but the notification window is consistent in stutter generation.
If you turn off GSYNC and run the monitor with a fixed refresh rate, the stutter is no longer present in Fullscreen mode (or any other mode).
Apparently, in the latest Beta for GameMaker Studio, there are fixes for high refresh rate monitors such as GSync so this may be a known issue: