Webpack is your compiler, and learning how to use that tools shouldn’t be the first thing you do, because a lot of people will probably give up before they get started. I think that most experienced developers should understand what the webpack configuration does. And all teams should have at least one person that can wrangle together a config file from scratch.
I say, start with create-react-app, then learn how to write react code, and then learn how webpack works when you have a thing you need to do that create-react-app can’t do.
I typically start with create-react app these days, and then move on to using my own webpack, if or when I need to.