I fished the Boston Harbor for years out of a 14' Starcraft with a 25 hp. I would go out to the Graves and beyond on a regular basis. Even went 15 miles out looking for tuna when I heard reports that they were in that close. I took it out of Ipswich and Newburyport, through the Merrimack and night fishing the beach front of Plum Island numerous times. Once I was going out of Winthrop during a small craft advisory and an older couple in a 23' boat were coming in, they started waving at me telling me to turn around. I just smiled and nodded and carried on. I never felt like I was in danger.
When I got my bigger boat (17' whaler), and took my wife out to the Graves for the first time she said she couldn't believe I used to fish out there in the tin boat.
You need a reliable motor, and I would not do anything less than a 25 as you need power to not get caught in the swells. Also, the boats are very uncomfortable for passengers sitting on the seats up front as their backs and behinds get banged up pretty good.
So... go for it and have fun.