Katie and the Dinosaurs, by James Mayhew (1991), because I am a pathetic reader in terms of what would make sense to read (instead of reading a nice time travel book over the weekend, I read some nice books that don't come out for months and months....
But in any event.
So Katie's grandma takes her to the museum to see the dinosaur bones, and promptly sits down to have a nap, leaving little Katie to explore on her own. Bad idea! Katie, after happily viewing all the fossils, goes down a dark corridor, opens a forbidden door, and finds herself in Dinosaur-time! There she meets and greets and sees from afar various dinosaurs, helps a hadrosaur find its family, and is chased by a T-rex.
It is an old book, so there are no dino feathers in evidence. It's also dino stew--with dinos who didn't live in the same time period all running around together. So though it feels written as a first guide to dinosaurs, it's better not read as science, but more as a merry little story for the very young (3 or 4) who won't become agitated by factual inaccuracies. So should you bother at all with it? There are many more accurate dino books around these days, I imagine, but the art here is rather charming, and the timesliping through a forbidden door is a magical twist of memorableness. Here's a sample page: