Does my bookkeeper need to be certified?

In one word: Yes. Having a certified bookkeeper is very important, especially if you are a small business owner. You will most likely end up saving money in the end if you pay a little more for a certified bookkeeper. With their training and experience, they will surely find any possible errors that might otherwise cost you in overpaid taxes and penalties.