There is a requirement for backwards compatibility and that is you have to have a hard drive for your Xbox 360. The 4GB Xbox 360 slim won't work for BC unless you put a hard drive in it. It also needs to be mentioned that you must use an official Microsoft-brand Xbox 360 hard drive. Third party drives you may find for cheaper on eBay do not have the necessary partitions to allow for backward compatibility, so it will not work (see full article here).
The way the backwards compatibility software is updated on your system is as simple as putting a backward compatible Xbox game into your Xbox 360, and the update will download from Xbox Live automatically.
Besides it being really convenient to be able to play both Xbox and Xbox 360 games on one system, playing Xbox games on your 360 has other benefits. Each and every compatible Xbox game will be upped to 720p/1080i resolution (assuming you have an HDTV) and will take advantage of full screen anti-aliasing.