Being able to follow the history of the created world (I don't need every detail but enough to get a general flavour of, say, how violent the world has been and whether it has moved on from there or not and why that is the case).
Being able to zoom in on specific details that move the story on and having a general overview (and seeing how that influences the specific details).
Having a reasonable sense of the geography and climate of the world (expecting these to have some impact on how people live their lives).