Smart glasses and AI-powered “phygital spaces” are transforming how people perceive, navigate, and interact with the world. Yet existing regulatory frameworks - Privacy by design and Human Rights by design - focus narrowly on data protection and overlook the relational and collective dynamics these technologies disrupt. This article introduces the ‘ethics of interactions’ as a complementary framework that situates regulation within the lived realities of AI-mediated environments. Our interdisciplinary team, combining law, speculative design, and human–computer interaction, developed a year-long methodology spanning speculative storytelling, legal mapping, field diaries, and role-play based workshops in Germany and Israel. These exercises revealed how smart glasses might reshape trust, consent and perception, through five recurring modes of interaction: person-to-person, person-to-space, person-to-reality, person-to-machine, and person-to-platform. Workshop role-play surfaced vulnerabilities that conventional legal analysis misses, from peer-to-peer surveillance and emotional inference to corporate, platform and government control of augmented realities. Building on these findings, we propose regulatory recommendations that combine dual governance models, interaction-sensitive safeguards, and investment in digital literacy for policymakers. By embedding speculative inquiry into policy design, this study closes the loop between abstract principles and lived dilemmas, offering both conceptual and practical pathways for governing phygital spaces.