While you should take your time exploring the regions, solving puzzles, and doing quests at your own pace, you should know that some events are locked to certain regions. This means that you have to be able to access those regions to participate in events for Primogems and other loot.
While it’s not the end of the world if you miss out on the rewards, you’ll want to get to Inazuma (the newest region, which unlocks at Adventure Rank 30), to be able to get the full experience.
Some characters from Inazuma also require materials that can only be farmed there, so if you get them from the gacha, you won’t really be able to use them until you get to their homeland.