Dirty power can be cleaned up by putting a resistive load on it, which will absorb the dirtiness. A 500w incandescent flood light will do.
http://www.f64.nu/photo/gen/image03.jpg shows dirty power out of my Kubota. A small load fixed it. Just little imperfections in the sine wave.
http://www.f64.nu/photo/gen/image05.jpg shows the waveform with the load. I didn't have the scope triggering right, but you can see the waveform is clean.
The UPS governor might be out of adjustment too. UPSs don't like 59.0 hz, they like 60hz. Frequency and voltage drop together on generators. I have had to make minor adjustments to the governors of many generators to get it just right under a steady load. A Kill-A-Watt is probably the cheapest way to measure frequency, but an APC smartups with a monitoring cable will work too.