Pantoja was born in San Juan, Puerto Rico where she received her primary and secondary education. She was later able to study at the University of Puerto Rico with the financial help given to her by her wealthy neighbors. There she obtained a teacher’s certificate in 1942. In 1944 she moved to New York City where she found a job as a welder in a wartime factory. She subsequently won a scholarship to Hunter College in Manhattan, where she graduated with a Bachelor’s degree in sociology in 1952. She then studied at Columbia University’s New York School of Social Work, where she earned her Master’s degree in 1954.
In 1957, Pantoja founded the Puerto Rican Forum (originally the Hispanic American Youth Association or HAYA), which served as an incubator for organizations and programs promoting economic self-sufficiency. This organization is now known as the National Puerto Rican Forum and is headquartered in The Bronx.