drill compensation for anything should be set at 110%-120% of the Bur RADIUS; so in a diameter 1.0mm finishing tool, drill compensation should be between 0.55 and 0.6mm. on a denture however this still leaves a lot of high spots and rough edges, and you can tell it leaves these by running a cotton ball around the rim of the denture once its out of the mill. there will be catches. due to spiral patterns and uneven surfaces, the surface quality will also have left many topical lines. you will not get those smoothed off with the finishing pass, it will take a specially devised second finishing pass to remove those lines. the majority of CAM don't really do that and i'd recommend it only on a 2mm intaglio finishing pass.
i'd bet that 3shape setting of 0.8mm was thinking the final finishing would be done with a 2mm tool.