Hello and Welcome! Congrats on your new pup!! Even if you have a fenced in backyard, its always good to take your new pup out on a leash for potty time, that way they learn they go out to potty first, eventually after your pup is older and has the routine totally down, they wont need you to be out there with them.
It's normal for a very young puppy to want to stay close to you, it's nature's way to keep them safe. As your puppy grows, so will its independence, provided you socialize it properly. I agree with Natalie about the leash for potty training. Also, pups tend to eat a lot of junk, it's safer for you to be out with the pup and make sure it does not do that. They grow so fast, enjoy this special time.