GO:0033608: formyl-CoA transferase activity
Catalysis of the reaction: formyl-CoA + oxalate = formate + oxalyl-CoA.