Confused

[SpeedoStocky]

I go swimming every week in a pool in my black speedos. Afterwards, an older english man also wearing black speedos came into the sauna and started talking to me. He's one of the only other guys I've seen wearing speedos in there. He asked me if I'd like to join him for laps in the future. We exchanged numbers in the changing room and sure enough, the following week, we went swimming again in our speedos.

This was great, we swam, chatted in the sauna etc. He then invited me out for drinks after. We spent way too long and got very tipsy, but then just went home. He has a wife and kids and hasn't really hinted that he wants again more from me. He continually invites me to come swimming, and we are always the only ones in speedos. What I'm confused about is, does he want anything more from me, or is he just looking for a friend, and maybe enjoys wearing speedos. Whenever I brought up the speedos in the pub, he would quickly change the subject, any ideas?