The other main penalty is Penguin, which looks at where the links to your site are coming from. A lot of people think they can spam low quality links to their site and get more traffic. If a lot of low-quality sites link back to you, it actually hurts your overall page ranking thanks to Penguin penalizing you. If you see odd traffic sources and notice that your page visits are going down, it’s likely that you’ve been hit by this.
Updating your site to recover from a penalty could literally take updating hundreds of pages, and possibly even re-designing the site from scratch. As such, many people take to hiring search engine and website experts to help them with the task. There are a lot of unknown variables with Google’s algorithms, and they change on a regular basis. If you care at all about your site, you should look into getting some help to make sure your site is fully optimized and as search engine friendly as possible.