Sims disappear from the family tree after the game culls them. Ghosts are removed from the family tree as if they never existed. The Sims 4 automatically starts deleting dead Sims from the family tree after the number of total in-game characters, townies included, has reached a certain threshold.
As a result, your Sim’s ancestors disappear from the family tree photos. A faceless grey silhouette remains in their place indicating there’s no Sim there.
That’s the way The Sims 4 works. It simply can’t handle too many Sims at a time. The game deletes Deceased Sims from family trees, and relationship panels to free up space.
Go to your game settings, and check what limit you set for your played Sims count. If you hit the maximum number or played Sims, don’t be surprised if the game starts culling.
Keeping too many played Sims on low-specs computers will make the game sluggish.
If you’re playing Legacy, you’ll be disappointed
If you’re doing the Legacy Challenge, it’s very hard to play a generational game when Sims keep getting deleted. This leads to very frustrating situations when twins are no longer related, grandchildren are cut off from their grandparents, and so on.
As Redditor Tarusama says, players should be the ones to choose which Sims get to remain on the family three and which get deleted.
I was playing with my legacy family and once I discovered this I got pretty disappointed. I hope they can give the option to keep the sims, in the future. If it slows the game down it can always be disabled, but the player should be able to choose.
Seeing Sims you’ve raised from a baby to old age cease to exist is almost unbearable.
Check if this is a known issue
If the Sims that disappeared were not deceased, this indicates the game is bugged. Check if this is a known issue or if it’s just you. If the problem is on EA’s end, submit a bug report and wait until they solve the issue.
As a workaround, load a save game where the problematic Sims were still present in the family tree.
By the way, if your baby disappeared after birth, try resetting the bassinet and visit a household you’ve never played before. Do that and then check if they appear in the family tree.