Swimming and Summer just go together. Whether it's a pool, a popular beach, or a remote swimming hole.
My favorite is usually the latter - though, I get my chlorinated pool fix as much as possible too especially with the heat wave-like temperatures. My hair still smells like pool from a wedding I went to this weekend at the YMCA's Camp Hough out in western NY.
So far this summer I've been to beaches, pools, and lakes but I haven't yet gone to a swimming hole.
MSN somehow made a list of top 7 such swimming locations in the country. Of course swimming holes in Hawaii and California made the list - but so did Ithaca, NY and Lincoln, NH.
The Lincoln, NH one, called Franconia Falls, is about 4 hours from the Albany area and at the end of a flat 3 mile hike that takes about 90 minutes. A description says there are chutes, and areas to jump into the water. I'm tempted to make the drive soon - maybe incorporating it into a beer trek with the bf.

And Ithaca's Robert H. Treman State Park is about 3 hours away with a lot of gorges thus also a lot of swimming holes. I had heard of this one before. It has a great location in the Finger Lakes - a perfect summer place to be with wine and water.
On the way back from Camp Hough, we stopped along Keuka Lake in the Finger Lakes as well. We didn't do any swimming but there was much wine and beer to be had. Luckily, I wasn't the DD.
And, if you go to Ithaca, you can turn that into a beer trek too - try the Bandwagon's spicy raspberry beer. It's one of the more interesting beer's I've tried.