Romelu Lukaku put the Nerazzurri ahead early in the second half, before a Ramy Bensebaini penalty levelled then Jonas Hofmann gave Gladbach an unlikely lead with six minutes to play.
Then in the 90th minute Lukaku bagged an equaliser from close range, to give Inter a point.
Earlier in the day, Real Madrid were beaten 3-2 by Shakhtar Donetsk, in Group B’s other match.
There was little in the way of goalmouth action in the first half other than a Lukaku opportunity 41 minutes in which the Belgian put just wide of the post.
Almost immediately after the break, Lukaku put the ball past Yann Sommer after a goalmouth scramble, which was his 10th goal in the Champions League, and his ninth consecutive European goal.
Inter’s lead was gone just after the hour mark when Arturo Vidal tripped Marcus Thuram in the penalty area, resulting in a spot kick following VAR review. Bensebaini stepped up and put Borussia level.
As the game went into the final stages, Lautaro Martinez smacked a Matteo Darmian cross off the post, and just moments later Antonio Conte’s men found themselves behind.
An excellent through ball from Florian Neuhaus found Hofmann, who had got in behind Vidal, before slotting the ball between Samir Handanovic’s legs and into the net.
The drama continued right to the death as Lukaku turned home an Alessandro Bastioni header in the final minutes.