Podcasts are just audio files (usually MP3 or OGG format) that are hosted on a web page or sometimes via FTP or BitTorrent. You can use whatever your favorite MP3/OGG player is to play the podcasts without any trouble. I usually use XMMS. It is very mature and seems to be pretty stable, robust and feature rich.

Most podcasts have an associated RSS (a website syndication format) file which is updated every time a new episode of the podcast comes out. Most people use what is called a feed aggregator to download the content that these RSS files point to, but usually podcasting sites have simple links to each episode displayed on their website. You can usually just go to their website and click the link to the episode that you want to hear and it will open up in the associated media player. You can visit the Linux Questions Podcast website to try this out. You should see a heading for each episode of the podcast and a link to the MP3 and the OGG format audio files under each heading.