I went over the derivation of MAP-MLE equivalence again and found my conceptual mistake. We are actually not computing the posterior probability over the parameters but the posterior density. Contrary to the probability, the density is not 0 in every point. Then you can indeed simply use Bayes rule and
throw out P(\theta) because it is constant (where in this case P(\theta) is a density and not a probability).
I went over the derivation of MAP-MLE equivalence again and found my conceptual mistake. We are actually not computing the posterior probability over the parameters but the posterior density. Contrary to the probability, the density is not 0 in every point. Then you can indeed simply use Bayes rule and throw out P(\theta) because it is constant (where in this case P(\theta) is a density and not a probability).