Kulej was born in Czêstochowa, Poland. At the Summer Olympics in 1964 he won a gold medal in the Light welterweight division (<63.5 kg) defeating Yevgeni Vasilyevich Frolov. In 1968, he defended his title in a close match against a Cuban boxer Enrique Regüeiferos becoming the only Polish boxer to ever win two Olympic gold medals. He also twice won a gold medal at the European Amateur Championships in 1963 and 1965, and won a silver medal in 1967. He had a record of 317 wins, 6 draws and 25 losses.