Our architect came over and we had a nice discussion with the neighbour about his concerns. Despite the fact that he didn't object to the plans, he still has some issues with the windows. So really if we wanted to, we could just disregard him completely, as he had his chance to object. But we have to live next door, we will do the neighbourly thing and at least take him into consideration without compromising the architect's vision completely.
All in all, it was a good discussion and there is a path forward.