If you want to go in November, go in November.  There's no guarantee on the
weather at any time of year.

I've been in November, December, January, February and April.  I loved it in
April but apparently I just got lucky.  The winds are usually high in the
spring and so the seas are quite rough.  However, in Dec, Jan and Feb, you
can run into nothers that sweep down and make it a bit chilly.

I enjoyed it in November.  There will probably be a bit of rain but I found
that it would rain only for a few minutes, sometimes an hour or so, with a
warm rain and then everything would clear up and be lovely again.