More pistols?

I currently own two pistols, a 9mm and .380. I'm not sure a .22 pistol would be worth owning, necessarily, so I dont plan to. I really like the Sig P365x, Glock 43x, and maybe a couple others. I also want to branch out into owning other types of firearms, namely rifles.

Should I continue with pistols for a while, or just start moving into rifles and come back to pistols afterward?