Fred Ascoli: The Harbor View

Fred Ascoli lives the life that many people who fall in love with Cape May have only romanticized. Like most native residents, his relationship with the ocean and his understanding of its ever-changing personality began as a child.

By the time he was a teenager, he was out to sea daily, working as a deckhand in the lucrative party boat industry. The very boat that he worked on later became his own. The nearly abandoned boat yard overgrown with weeds and dilapidated shacks also became his and was transformed into one of the most popular and scenic Cape May dining destinations, Harbor View Restaurant.

True to its name, Fred’s Harbor View offers a sweeping, unobscured view of the marinas, waterways, fishing boats, and sunsets that have kept Cape May a favorite among boaters, nature lovers, and artists for generations. Connoisseurs of fine seafood are astounded by Fred’s menu selections which are regarded by regulars as among the most extensive and creative on the Jersey shore.

Let’s start by talking about your life on the water.

I’m a licensed captain. I used to own the party boats Miss Chris and Lady Chris in Cape May. I sold them both about four years ago. I first went to work on the boats in 1979 and I bought them around 1985 from Captain Frank Spiegel.

Did you only take people out fishing in the summer?

I got to the point where the boats fished almost year-round. There were more species of fish available back in those days. In the fall we used to wreck fish and I began doing offshore wreck trips for sea bass, codfish, and pollock. I did those as 12-hour trips, and that was done through the winter until the mackerel came around at the end of March and April.

How many people would you take on a trip like that?

I would take up to 50 people. We had a pretty good reputation and we had people coming from all over to fish with us, so the clientele were pretty much seasoned fishermen.

You must have a wealth of memorable moments being out there all the time with so many people.

There were so many things through the years. In terms of fishing, we got the state record for sea robin, we got the state record for spiny dogfish. My boat, which Captain Butch Boyd ran at night, caught the largest bluefish in the state. I remember he caught that bluefish out on the bank. The guy who caught it knew it was a big fish, but he didn’t know it was a state record. The guy took it home, put it in his bathtub, and packed it with ice.

At what point did you go from being a fishing boat captain to running one of the most successful restaurants in the county?

Around 1995 when the condominium boom was happening, the Miss Chris and Lady Chris were kept over on Schellenger’s Landing. Spiegel wanted to sell the property and he wanted to sell it to me. And I had absolutely no money. Nothing was here on this spot. I bought this property with the idea of bringing my party boats here because where they had been before was sold and was going to be condos. I came up with a plan and designed the marina with an engineer. Got all my permits, which took quite a while. We got the tackle shop running and that took off really well.

How did the restaurant develop?

So many people would show up really early in the morning. They’d drive from Philadelphia or upstate Pennsylvania. They’d get here at 5 or 6 in the morning and load their stuff onto my boat and then they’d go into Cape May for breakfast. So, I figured I’d start the restaurant. What’s now my bar used to be counter dining for breakfast. One day, a real estate agent walked up to me and said, ‘I represent so-and-so and they have a liquor license, and they’d like to know if you’d be interested in buying it.’ I asked him how much, called the bank and I said, ‘I need a little more.’ That’s how I ended up with the license.

Restaurant owners would kill for an opportunity like that!

They would. That’s what turned this from counter service to a bar. When I built the outside deck over the sunken ferry, I went back to the planning board and put a bar and grill out there. Back in the 90s, there weren’t a lot of outdoor eateries other than the Lobster House, where you could eat on the docks and the Schooner.

When you captained the Miss Chris 30 years ago, did you ever think you’d be running a successful restaurant and bar?

Absolutely not. And when I look forward to the next 30 years, I wanna be hunting and fishing.

How has the party boat industry changed since you started in the 1970s?

Fishing regulations are completely different. There are now so many bag limits and size limits. Back in the 80s, we had mackerel to fish for. Cape May even had a whole charter boat fleet that would troll bluefish in the spring. In the fall you could go wreck fishing to catch sea bass. Cape May was known all up and down the East Coast for stripers. That’s all gone today.

Why?

There are all different kinds of theories as to why. My theory is that things run in cycles. There are indeed natural cycles. But I also think that man has something to do with it. There was a time in the 80s when all the fish were down here along Cape May. In North Jersey, you could hardly catch anything. Now it’s the opposite, there seems to be hardly anything here and it’s all up north.

Was there ever a time someone on your party boat pulled in a fish that was very unusual?

Oh, sure. There was one time I had a garfish, which is usually found much farther south. One time I had a guy who caught a sunfish (considered one of the largest fish in the world and weighing anywhere from 500 to 2,000 pounds). I begged the guy to let me just cut the line. The guy said to me, ‘Oh, no, captain, he bit my hook and he’s mine.’ I told him we couldn’t put the fish on the boat for several reasons, one of which was that there was no way to physically lift it onto the boat. He finally let me cut the line.

Is there a network of tips within the fishing industry as to where the fish are biting?

Back in the day, the bunker boats used spotter planes that would see the schools of bunker and then guide the boats to them. One fall we were southwest of the inlet and one of the bunker boats came on channel 66 and said, ‘Are any of you party boats down by McCrie Shoal?’ I answered back that I was about 3 miles from it. He said, ‘We’ve got a big school of black drum here and I can put you on ‘em.’ I told him I’d be there in 15 minutes. I started the engines up and didn’t tell any of my passengers what was going on. Captain Bobby Schumann of the Sea Star heard the call and he said, ‘I’m going to join you, Fred.’ We started to get closer, and Bobby and I were coming closer, bow to bow. When we were about a quarter mile away, the spotter planes told us to slow down. All of a sudden, in front of me, the entire ocean turned a reddish bronze color. All the drum fish were swimming at the surface of the water, there must have been thousands of them. That color was the color of the fish. I saw nothing on my fish finder but could see them with my eyes in front of me. Then the color was gone, all the fish had gone under. I blew the horn; everyone dropped their lines, and each passenger went home with a 50-pound drum fish! That was a once-in-a-lifetime experience.

Do you miss those days?

Oh, yeah, that was the greatest job on the planet. It was the only job in the world in which no two days are ever the same. And you’re only as good as your last catch. That means if I had 50 people on my boat and we only caught 25 fish, I wasn’t very good, was I? I’m the captain so some people think that I am to blame. But most of the seasoned guys know that you can’t get ‘em every time.

Have you ever been involved with the rescue of someone in danger?

There was one time I was wreck fishing down in the mouth of the bay. I heard a Mayday call. ‘Mayday, Mayday, Mayday,’ and then nothing. The Coast Guard came on channel 16 and was calling for the vessel that was giving the Mayday. I could tell the Coast Guard was struggling. I knew a gillnetter at the time who was gillnetting up the bay. I called him and asked if he heard the Mayday and he said he hadn’t. I had one of those old mobile phones that came in the big bag with the shoulder strap, I plugged it into the cigarette lighter, I called the Coast Guard Base, and told them that I thought I knew where that Mayday was coming from. I gave them the position that I had plotted on the radar, and they thanked me. Later that afternoon I got a call from the commander who said when they got there, they were waiting a quarter mile from the boat. I was eight miles from the boat. The commander told me that unfortunately one person died but said my call had probably saved the lives of the other three because they were all in the water.

I know you’ve created a lot of the dishes here at Harbor View. If I were coming here for the first time and wanted to try your signature dish, what would I order?

That’s a tricky question. I was featured on the Outdoor Channel and the guy who was filming the segment said he wanted to try the Ahi Tuna Chips, and he loved those. He tried one of my clam pot pies; he took one bite of it, and said, ‘My God, how did you come up with this?’ I’m famous for my clam pot pies and scallop pot pies. I even serve rabbit pot pies. I only do the pot pies during the winter because I wouldn’t be able to make them fast enough in the summer.

What is your favorite thing about creating these dishes and running this restaurant?

It’s a lot like running the boats and going fishing. When I took people fishing, I got a lot of enjoyment from people catching fish. When people come in here and eat and they love what they are eating, it gives you that sense of “mission accomplished.”