These "line numbers" are part of a subtitle sequence.
That is, Subtitle Edit treats them as part of the text, not an actual new line, or more accurately, sequence.
I don't know if @VelaOmsk's suggestion works.
@forgetit's and @TestoteronnnieFlex's approaches won't work, I'm afraid. @forgetit's suggestion to use notepad++ does remove one of the duplicates.
I know nothing about regex, but, for what I gather, in order to remove these duplicates one would require some code doing something like this:
"Remove all consecutive lines of text that contain only digits
up to the one that has as next line an empty one. (or back-to-back line breaks)"
This might remove more than what you asked for, but could also work 🤷♂️: in Subtitle Edit, go to Edit -> Replace, put
into Find what: and leave Replace with blank. Select Regular expression (not something I'm particularly knowledgeable about) and replace all.
Then a Fix common errors with Remove unneeded spaces should get rid of the left over line breaks.
EDIT: that's also a pretty bad regex, TBH, because I don't know regexes. Maybe add a \d? to the end if it misses some