When a baby is growing inside its mother, it's not able to breath or eat like you do instead, it gets its air and nutrients straight from Mom through a tube connected to its belly. This tube is called an "umbilical cord," and is where our belly buttons (also called "navels") come from.
After the baby is born, a doctor will cut the umbilical cord since the baby can now breathe and eat on it's own. (Don't worry, there are no nerves in the umbilical cord so cutting it doesn't hurt the baby!) Within a few days the stump will dry up and fall off on its own, and what's left behind is the baby's very first scar: a belly button!