Start with the finest grit possible. Start finer than you think you need and if that won't cut it step up a grit coarser. There is a fine line on where to start. Starting one step too coarse can actually take more time and work to get the deep scratches out than starting one step too fine. On the other hand starting a couple steps too fine can take forever or never even go deep enough to take out the imperfections.
I would never consider 80 grit unless it had some crazy deep pitting from sitting at the bottom of the ocean for a decade. Even 180 seems uncalled for IMO in most situations.
For something that looked good and was reasonably smooth I would try to start around 400 grit. If that didn't cut it I might step down to 180 or 220.
More importantly power tools will change your life. I don't know what you have access to but take advantage of what you got.
One of my favorite tools is my Pneumatic angle grinder.
Something like this.
Now I would try to start with a fine grit nylon roloc which should be the blue one.
I would quickly hit the whole surface and work out what I could easily with that. If there were areas that it didn't clean up nice I would go to the next coarser which would be the maroon. If that didn't take out areas I would hit just those areas with the brown (coarse).
Now if there were areas with deep pitting I would step down to a flap disk but use the finest possible and only in the areas that really needed it.
Once you can no longer see any imperfections at the grit you are at move up to a finer grit. If that reveals any imperfections clean them up with a grit lower but only hit that area then go back to the next finer grit.
This may sound a little bassakwards but I would be willing to bet that 90% of the DIY polishes that look good in pics will still have visible sanding marks from coarser grit abrasives and the few that don't probably have a lot of time put into getting rid of them. I can tell you it sucks when you get down to 600 grit to notice 80 grit scratches that won't be taken out by anything finer than 220. Then you go back and start all over again.
If you can use the roloc mesh wheels from the fine disk especially if you do your final round with one pretty worn down you can jump to about 400 grit.
The other option is to use buffing bobs and wheels and jump straight to a very coarse compound. Mcmastercarr has some very coarse compounds that will allow you to jump from a pretty rough finish to polishing.
What I do is wear out a nylon roloc pretty good and then add compound to that and use that as the buffing bob. This makes a pretty good transition from sanding to polishing. Once you get most the ugly stuff worked out you can move to a proper bob or buffing wheel and compound.
This would probably cut your time by 50% to 75% over doing it with a sander and hand finishing and leaves a lot of the heavy effort up to the grinder instead of you.
It can also get into much tighter areas than your average sander.