I recently upgraded from an iPhone 4 to the iPhone 5 as my contract was up. The differences are mainly aesthetic in that the phone is taller with the new bigger screen. It is however the same width as the 4, though the extra screen real estate is nice and gives the 5 a bigger feel which is still usable with one hand.
Speed is another big difference I have noticed. Apps on the 4 took a time to load whereas on the 5 they load almost instantly in most cases. Overall the 5 feels a lot more snappier, for example, the camera app loading to taking a photo is very quick compared to the 4. I used to have to ask people to wait or I missed the photo opportunity completely. Not good when you have small children performing funny Kodak moments!