The only problem when disabling VM is with memory leaks. You may suddenly end up with 0 free space and only take notice because Windows stops responding. With VM enabled you notice it earlier because of the constant hard drive sounds

and if you're lucky enough you'll get a Windows notification informing you that the page file is getting larger.
Another problem is that some games require VM to be enabled (one game required me to have 1GB before it would launch!!!) even if you have 10 times the max amount of memory the game will ever need.