Federal Credit Unions are essentially banks that are owned by their clients instead of publicly traded or what-have-you. Instead of being part of the FDIC, they have the National Credit Union Association (NCUA). They tend to be able to offer higher interest rates on savings and lower interest rates on loans than banks can, due to their mutual-ownership structure.
Credit Unions operate as non-for-profit businesses, which can allow their management to use 457 retirement plans, but they are not associated with the Federal government. They do, however, charter under federal regulations, as opposed to state banks.
The NCUA insures the deposits of clients the same way that FDIC insurance does. Many credit unions have not expanded their products to include the kinds of commercial loans (and other kinds of loans) that banks make, but some are slowly venturing into that territory.
Because the profits of the business are shared among the clients, credit unions can often offer higher interest rates on savings and CDs than banks, as well as lower interest on loans.