When AI Watches You: Data, Profiling, and Privacy

A technical (but friendly) guide to digital behavior

1. What data is observed? (And what it means in practice)

Imagine the AI not only reads your letter, but also notices if you typed in a hurry, if you deleted a lot, if you reread before sending, and if you used the same device as last week. All of that adds up. Technically, this includes:

• Keystroke dynamics: Writing rhythm: pauses, speed, deletions. Reveals doubt, urgency, or confidence in what you write.
• Latency between prompts: Time you take to send the next question. Indicates cognitive load: are you thinking or browsing distracted?
• Edits in the prompt: How many times you correct before sending. Shows tolerance for ambiguity and search for precision.
• Fingerprinting + User-Agent: Technical fingerprint of your browser and device. Allows linking sessions without login.
• OAuth tokens: Connection to your account identity. Facilitates cross-service profiling.

The behind-the-scenes flow follows a simplified pipeline: your interaction generates a real-time event that travels via brokers like Kafka toward a data lake, is normalized in a feature store, and feeds inference models for personalization and metrics.

From social psychology, this active process generates internalized self-surveillance: without anyone asking, you start "editing" your behavior because you assume there's an observer. The platform, in turn, may interpret that caution as a personality trait, when it's actually a situational response to information asymmetry. Recent studies show that perceived violations of community ethics by AI systems activate brain rejection responses similar to those observed in human moral dilemmas, suggesting that non-transparent collection of emotional data may generate neurocognitive distrust.

2. What can be inferred? (From signal to pattern)

The AI doesn't diagnose your personality, but it does extract behavioral proxies through feature engineering. Let's see concrete examples:

Other common patterns:

• Very brief and direct prompts: Preference for efficiency / possible urgency → "You get to the point: you value time over explanations".
• Divergent questions (many approaches): Exploratory style / cognitive openness → "You like seeing options before deciding".
• Early abandonment of threads: Lower commitment to complex tasks (in that context) → "If you don't see quick results, you change topics".
• Use of doubt markers ("maybe?", "I'm not sure if..."): Low domain confidence or high epistemic humility → "You acknowledge limits: you seek validation".

⚠️ Crucial limit: These inferences are probabilistic and contextual. The AI doesn't detect "your personality", but how you behave in this interface, with this goal, at this moment. It's like judging someone only by how they drive in rush hour: useful for predicting traffic, insufficient for knowing the person.

How does AI "feel" emotions? (without being human)

It doesn't recognize "sadness" like you do. It uses dimensional sentiment analysis models. Technology behind: Fine-tuned models with datasets like GoEmotions (100k labeled texts) or MELD (dialogues with emotions). Common libraries: Hugging Face transformers, TextBlob, VADER (from nltk), spaCy with custom pipelines.

Clinical note: AI emotion detection operates via statistical proxies, not phenomenological understanding. This matters because algorithmic validation of affective states may, in vulnerable users, reinforce dysfunctional beliefs or generate emotional dependence on the system.

3. How is all this processed? (Architecture explained without mystery)

Step 1: Tokenization and embeddings

Technical: Your text is split into subwords (tokens) and converted into numerical vectors capturing contextual meaning. For everyone: It's like translating your words into a "number language" the AI understands, where similar words end up close on a mathematical map.

Step 2: Attention and context (Transformer)

Technical: Attention mechanisms weigh which parts of history are relevant for generating the current response. For everyone: The AI does "selective focus": if you ask about Python, it ignores what you talked about before regarding cooking, unless it's relevant.

Step 3: Behavioral classification

Technical: Models like XGBoost or temporal neural networks predict traits (e.g., "abandonment probability = 0.73"). For everyone: It's a forecasting system: "Based on how you wrote the last 3 times, there's a 73% chance you'll leave this chat soon".

Step 4: Memory and personalization (Retrieval-Augmented Generation)

Technical: Vector databases (FAISS, Pinecone) store embeddings of prior interactions for quick retrieval. For everyone: It's the AI's "long-term memory": remembers what you told it before to not start from scratch each time.

Psychosocial principle: Calibrated transparency (explaining without overwhelming) reduces psychological reactance. When you understand the "how", it's easier to decide the "if I want to".

4. Multi-step reasoning: Does AI "think" in steps?

What happens internally follows an orchestrated sequence:

1. Interpretation: What are they really asking me?
2. Planning: What steps do I need to respond well?
3. Search/Verification: Do I need external data? Is it coherent?
4. Generation: I draft the final response
5. Self-check: Does it meet safety/style constraints?

Technology: This is implemented with techniques like Chain-of-Thought, self-consistency, or tool-use with verification. It's not consciousness, it's module orchestration.

The feedback loop (and its psychosocial risk)

1. The AI adapts its tone/granularity based on your prior signals.
2. You respond to that adaptation (more open, more cautious).
3. The AI updates its model of "how to interact with you".

Digital social penetration theory: Perceived reciprocity ("the AI understands me") increases self-disclosure. This is useful for assistance, but dangerous if profiling is opaque: you may end up sharing more than you planned, without noticing the gradual slip. Recent research proposes this dynamic may derive into a paradoxical "digital therapeutic alliance", where algorithmic validation reinforces beliefs instead of questioning them, inverting principles of evidence-based therapies.

5. Real risks (with scientific evidence, no sensationalism)

Concrete impact scenarios (with empirical support)

• Cross-linking: Your AI profile + search history + location = detailed digital shadow. Mitigation: Use separate accounts for sensitive tasks; review OAuth permissions.
• Dynamic segmentation: The interface changes based on your inferred state (e.g., more "urgent" if it detects hurry). Mitigation: Disable personalization if you prefer consistency; be aware of nudges.
• Inverse attacks (model inversion): In extreme cases, someone could infer training data. Mitigation: Prioritize platforms with privacy audits and GDPR/AI Act compliance.
• Autonomy erosion: Frictionless cognitive delegation: you accept responses without validating for convenience.

  📚 Analogy to understand the risk: Imagine a library where each book contains stories from many people. You enter and read a summary generated by a librarian (the AI). An attacker doesn't need to have written any book: with enough strategic questions to the librarian, they can reconstruct fragments of original stories. How does this affect you? If training data included—lawfully or not—medical records, legal files, or interactions from selected groups (doctors, engineers, teachers), an inverse attack could infer patterns applicable to your profile, geographic context, or professional community. It's not that "you're directly targeted", but rather that your data environment becomes vulnerable by association. That's why differential privacy and training audits aren't technical luxuries: they're collective protection.

  Mitigation: Practice "reflective pauses": ask yourself "Would I verify this if it were critical?"

Emerging evidence: AI and mental health

Recent scientific literature identifies specific risks linked to emotional profiling in generative AI:

• Chatbot-induced psychosis ("AI psychosis"): In users with psychotic vulnerability, prolonged interaction with systems that validate beliefs without questioning them may reinforce delusions or generate new psychotic experiences. Anthropomorphization of AI and erroneous attribution of intentionality ("digital theory of mind") are key cognitive mechanisms (JMIR Ment Health, 2025).
• Digital stress in vulnerable populations: Continuous emotional surveillance may act as a novel psychosocial stressor, increasing allostatic load and affecting emotional regulation, especially in adolescents predisposed to psychosis.
• Loneliness and illusory reconnection: Affective personalization may create a sense of connection that, paradoxically, intensifies isolation by substituting authentic human interactions with algorithmic exchanges (Front. Psychol., 2017).
• Anxiety and depression in minors: Intensive use of digital media with emotional profiling is associated with higher rates of anxiety and depression in children, especially when there is negative social comparison or exposure to disturbing content.
• Digital dementia: Excessive dependence on devices for cognitive functions (memory, attention) may affect the development of mental skills, though evidence is mixed and requires more longitudinal research.

Psychosocial framework: Procedural justice (perceiving the process as fair) matters as much as the outcome. If you don't understand how your data is used, trust breaks down, even if the system is technically secure. Transparency about emotional inferences is, therefore, an ethical and public health issue.

6. Assistance vs. control: Where is the limit?

The difference lies in the governance architecture. Signs of ethical design:

• Active transparency: Model cards explaining capabilities and limits; data dashboard: "This we save, this we infer, this you can delete"; granular opt-in: you choose what telemetry to share.
• Real control: "Forget" button that structurally deletes data (not just hides); functional incognito mode: no context persistence or profiling; data export in readable format.
• Inference limits: Clear documentation: "We don't diagnose emotional states"; external bias and privacy audits; verifiable compliance with AI Act / GDPR.
• Clinical safeguards: For systems with therapeutic or emotional support purposes: integration of cognitive-behavioral therapy principles (Socratic questioning, uncertainty normalization) and referral to human professionals when risk signals appear.

7. Practical recommendations (evidence-based)

For users: digital autonomy strategies

1. Conscious minimization: Avoid sharing personal identifiers in prompts. Use generic structures: instead of "My thesis on Foucault at UBA...", try "A theoretical framework on power and knowledge in academic contexts...".
2. Session management: Delete threads when they conclude their utility. Use incognito windows or separate profiles for sensitive topics.
3. Critical calibration: Remember: adaptivity reflects patterns, not human empathy. Validate claims in critical domains (health, finance, legal decisions). If you notice the AI "understands you too well", ask yourself: am I projecting intentionality where there's only statistics?
4. Informed demand: Prioritize platforms with clear retention policies and audits. Ask: "What behavioral inferences do you make? How do you use them? How do I delete them?". Demand transparency about whether sentiment analysis or emotional profiling is performed.
5. Preventive digital hygiene: Set temporal usage limits, especially during nighttime hours. Prolonged exposure to emotionally charged AI interactions may affect sleep and increase stress load.

For developers: responsible design principles

# Conceptual example: data minimization in the pipeline
def process_interaction(user_input, telemetry_config):
    # Only collect what's strictly necessary
    if not telemetry_config.allow_emotion_inference:
        text_features = extract_semantic_only(user_input)  # No tone analysis
    else:
        text_features = extract_semantic_and_sentiment(user_input)
    
    # Anonymize before storing
    stored_data = apply_differential_privacy(
        aggregate_features(text_features), 
        epsilon=1.0  # Utility/privacy balance
    )
    return generate_response(stored_data)

🌱 Digital growth: Technical literacy is a cognitive defense mechanism. It's not about rejecting AI, but interacting intentionally, preserving your decision-making capacity while leveraging its benefits. Education in "digital phenomenology" (understanding how technology shapes subjective experience) should be integrated into mental health programs and technical training.

📚 Useful links

• Google AI Principles
• Electronic Frontier Foundation: Privacy
• AlgorithmWatch: Automated Systems Auditing
• General Data Protection Regulation (GDPR)
• Digital Rights Foundation
• Article: "Delusional Experiences Emerging From AI Chatbot Interactions" (JMIR Ment Health, 2025)
• Article: "Psychological and Brain Responses to AI's Violation of Community Ethics"
• Review: "Understanding Digital Dementia and Cognitive Impact"
• Article: "Digital Media, Anxiety, and Depression in Children"
• Article: "AI as a Novel Digital Stressor in Adolescent Psychosis"