I have been wearing speedos around my girlfriend and my kids for years... pool, beach, boating. No issues at all. Usually it is fairly conservative and the smallest are speedo solar.
Last weekend for the first time I wore speedos when i took the kids to my parents house and swam in their pool. I just don't see any issue with it...nobody said anything about my speedos.
Just wear them everyplace you would swim. I see no reason not to.
trying to attach a pic of me and my girlfriend wearing the same speedos
http://www.mensswimsuitboard.com/for...1&d=1464889165