Well it's a file format that can store audio data. It also seems to do this thing where it can store multiple tracks but I'm not too sure. I was actually wondering what this is used for myself so I'm glad someone asked.
Erm, well, there's always Google and Wikipedia. Might wanna take a peek there.
Short summary is that it's a popular open source file format that is capable of compression comparable to MP3, sometimes with marginally better quality (depending on source material and quality settings).
It's just another audio format like mp3 or m4a. It's widely accepted as a standard in the open source world, but not in the commercial world since Windows Media Player and iTunes don't support it natively. Also, I should probably mention that Vorbis is the encoding scheme, while OGG is the container format. Vorbis supports up to 255 channels of audio and up to a 192kHz sampling rate. IIRC, mp3 only supports up to two channels and up to 48kHz.
Since it can have better quality than .mp3 files, .ogg files are sometimes used for soundclips/songs in low-budget games, where the developers want to save space.
I believe most, if not all, of the audio in Minecraft is in .ogg format, for example.