Firstly ,it is a very small place and you can complete your trip in like 2 nights and 2you are travelling from Vellore there are always local government bus are available every hour.
I went to Pondicherry with 5 friends of mine in January.I think November-February is the best time to go and chill there.
We left at 5 AM by bus from Vellore and reached Pondicherry at around 9:30 AM. From bus stand we directly went to Mission Street and hire 3 two wheeler(₹200-₹400 per day) fromcan hire two wheeler from anywhere in Pondicherry. We booked a room in a hotel in Mission Street in White town.
Coming to the places to look around in Pondicherry-
Enter White Town and you will find a lot of french buildings there ,like a Frenchare very beautiful. I would suggest that park your vehicle somewhere and just walk through the white is totally worth only you will find Aurobindo Ashram and it is just blissful.
Then go straight to Rock Beach.The view from there is justand just relax on the rocks and listen to the sound of curling waves when it hits the is breathtaking.
The Basilica of the Sacred Heart of Jesus , this is very near to rock beach approx 1is very nice and very old.
Then straight head to Paradise Beach in your beachis about 13 km from town and trust me the ride between town to paradise beach is beach is calm and located in a remote area.
After this come back to your room freshen up and get ready forare lot of good places in white town and after dinner go on a walk again at the rock beach and visit and La Cafe. It has perfect beach there and have coffee.
Main point of being in Pondicherry is cheap alcohol (tax free alcohol),so grab as many bottles as you can of your favouriteare two known clubs in Pondicherry ,one is very far from the town and other one is costly. Sadly, there is not much club life there.
Then for next day wake up early and head to AUROVILLE. It attracts tourists from all over theyour whole day there and literally you will never forget this experience.Inside auroville there is MANTRIMANDIR. If you want to enter inside that there is pre-booking which has to be done (2 days before). It will be a wonderful experience.
This is all from my side ,Pondicherry is a very beautiful destination if you are looking for a short vacation.