what would happen if the elves did not leave middle earth?
Some elves did not leave Middle-earth. As the centuries passed, their spirits consumed their bodies, and they became invisible to mortals.
Note that these were mainly Avari. All the Noldor left Middle-earth at the end of the Third Age.
Yes, It would have been almost entirely Avari and some Teleri Elves who would have remained in Middle-earth to diminish and fade. We don't know for certain if Elrond's sons took ship to the West, but it seems most likely.