If you have an iPhone, this app comes built into your phone. It might not have the biggest feature set, but it offers a quick and easy way to jump into the world of podcasts. It lacks the more powerful features found in other players like "voice boost" (which is great for listening in loud environments like a subway train or an airplane) and the ability to skip silence in a podcast. However, you can stream and download shows, skip ahead or rewind in 15 second intervals, and set your favorite shows to download automatically as soon as a new episode is released.
Google Play Music
If you're on Android, Google's music streaming app, which comes preinstalled on your phone, also supports podcasts. Like Apple's Podcasts app, Google Play takes a simplistic approach to podcasts. It lets you subscribe to podcasts, download episodes, and skip and rewind during playback, but that's about it. However, there's also a version of the app that runs in the web browser, so you can listen from your computer when you're at your desk and save your phone's battery.