GO:0051562: negative regulation of mitochondrial calcium ion concentration
Any process that decreases the concentration of calcium ions in mitochondria.