What is a Bitcoin Wallet
A bitcoin wallet, while it is similar to a physical wallet in that it allows you to store your bitcoin, is more like your own bank which allows you to open and manage accounts, transfer, spend and receive money through the bitcoin network, and provides security.
Types of Wallets
Wallets come in three forms, software, hardware, and paper.
A software wallet, the most common, allows you to store manage your bitcoin from your computer, phone or website, and offers only basic security at this time.
Hardware wallets perform the same function, but on a dedicated device that can be disconnected from the network and is much more resilient to attack because it only performs the function of storing bitcoin, unlike a phone or computer which runs many apps and can be attacked in a variety of ways.
A paper wallet is the most secure, because it's not actually connected to the network and can not perform any function other than storage, until it's keys are added to a software wallet. These are useful, because an address can always receive and store bitcoin, only it's private key must be kept secure in order to properly secure the bitcoin - therefore, an offline paper wallet can be used to store and receive bitcoin in a fundamentally secure way.
Wallet vs. Address
Bitcoin is actually stored in addresses, not wallets. Think of an address like an account at a bank that has an account number and can receive or send money. Likewise, a bitcoin address can be used to receive, store, and spend money.
A wallet facilitates this functionality, although a wallet is not a part of the bitcoin protocol, it's merely a tool to enable you to use it. The wallet allows you to create addresses (like bank accounts) which is essentially an account number, and then use those wallets to create transactions to spend your bitcoin, or just store it until needed.
Within the wallet, all of your bitcoin is aggregated so that it can be spent together. You can still see the amount of bitcoin in a specific address if you need to with some tools, but the wallet figures out which addresses to use in order to spend the money with the least fees.
The paper wallet from above is basically just a paper address.
Because a wallet has the information needed to spend your money, if lost or stolen your bitcoin will not be accessible. Therefore it's important to always keep backups of your wallet, and all copies must be kept secure.
Check bitcoin.org's list of wallets here, which includes information about the security and potential concerns about each type of wallet.