RUGBY, England—Stuart Field raised his first flags in July: the English red cross of St. George and also a smattering of the British red, white and blue, hanging them from his hometown’s streetlights.

Aside from soccer or royal events, Brits don't fly their flag as much as Americans do. Now, with anti-immigrant groups embracing the Union Jack, it's part of a debate on what it means to be British.