Rav Yosef Tzvi (Joseph) Carlebach was born in Lübeck on 22 Shevat 5643 (30 January 1883), the eighth of the twelve children of Rav Shlomo (Salomon) Carlebach, rabbi of Lübeck, and Esther, daughter of his predecessor Rav Alexander Sussmann Adler. Several of his brothers became rabbis, among them Rav Hartwig Naftali Carlebach, father of the singer Rav Shlomo Carlebach. He studied Talmud with Rav Mordechai Gumpel, a disciple of Rav Yaakov Ettlinger, but according to Hevrat Pinto it was his father who influenced him most.

From 1901 to 1905 he studied mathematics and natural sciences at the University of Berlin while attending the rabbinical seminary of Rav David Tzvi Hoffmann. From 1905 to 1907 he taught science at the Lämel School in Jerusalem, where he came to know Rav Shmuel Salant and Rav Avraham Yitzchak HaKohen Kook; he later called those the happiest years of his life. He received a doctorate from Heidelberg in 1909 for a thesis on the mathematics of the Ralbag, and rabbinic ordination in 1914.

During the First World War, assigned to educational work in occupied Lithuania, he founded a Hebrew secondary school in Kovno modelled on Torah im Derech Eretz. After his father's death in 1919 he served as acting rabbi of Lübeck, and from 1921 to 1926 headed the Talmud Torah school in Hamburg, which he modernised, introducing Hebrew as a living language. Elected chief rabbi of Altona in 1925, he became chief rabbi of Hamburg at the Bornplatz Synagogue in 1936.

Under Nazi rule he went on preaching openly, even with Gestapo officers present. On 6 December 1941 he was deported with his family to the Jungfernhof camp near Riga; according to his daughter Miriam, he turned down an offer to stay behind so as not to abandon those with him, and secretly organised lessons and a Chanukah celebration in the camp. On 8 Nisan 5702 (26 March 1942) he was murdered in the Biķernieki forest together with his wife Charlotte and their three youngest daughters. His burial place is unknown.

Works: Lewi ben Gerson als Mathematiker (1910), Die drei grossen Propheten (1932), Das gesetzestreue Judentum (1936).