German has four unique characters, Ä/ä, Ö/ö, Ü/ü, and ß (called long s, or sz) which doesn't have an uppercase. None of these are considered part of the alphabet (unlike the Danish Ø), so if you're in school in Germany you say the alphabet from A to Z, then you say these at the end "ä, ö, ü und ß". In a library you'll find book titles starting with Ä mixed in with book titles starting with A.
The umlaut characters (ä, ö, ü) indicate a change in sound, and shouldn't be confused with the diaeresis which indicate a break in a syllable (usually in French and Spanish). In German, these characters began in the 16th century began using using the vowel paired with an E (ae, oe, ue, like seen in some old German last names), but over the centuries the e shifted above the letter, then slowly faded into the two dots we see today.
The ß is a symbols formed by mixing the old s, ſ, (as seen in the word Congreſs), and a cursive z. This pair of symbols were mixed together until it became a symbol of it's own.