Review ID: 5c5db522939eGenerated: 2026-04-21T19:09:53.064Z
CHANGES REQUESTED
81
Total Findings
23
Critical
28
High
29
Medium
36 of 108 Agents Deployed
DiamondPlatinumGoldSilverBronzeHR RoastyFree Baseline
Agent Tier: Gold
MemPalace/mempalace →
main @ 87102fb
AIAI Threat Analysis
REAL THREATS
Shell Injection & Command Execution (Critical)
• Shell command injection via eval in hook scripts (2, 3, 45-54) - Direct code execution from untrusted JSON input
• PID file race condition enabling arbitrary command execution (16) - Classic TOCTOU vulnerability
• Shell hook wrappers enable command injection chains (29, 30, 89) - Multiple injection points in hook system
• Path traversal in transcript path expansion (55) - File system access outside intended directories
Multi-Tenant Data Isolation Failures (Critical)
• Global backend instance cache shared across tenants (5, 6, 26) - Data leakage between users
• Entity registry shared across tenants (11, 14) - Cross-tenant entity visibility
• Knowledge graph queries lack tenant filtering (15, 17) - Information disclosure
• Palace export includes all tenants' data (12, 13) - Mass data exfiltration
• Deduplication operates across all tenants (9, 10) - Privacy violation
• Configuration system lacks tenant isolation (8) - Settings leakage
SQL/NoSQL Injection (High)
• Unvalidated parameters in where clauses (75, 76, 95-98) - Direct SQL injection vectors
• Raw SQL query construction without parameterization (111-113) - Database compromise
• Dynamic predicate construction without validation (84) - Query manipulation
Server-Side Request Forgery (High)
• SSRF via ChromaDB PersistentClient with malicious palace_path (60) - Internal network access
• SSRF via user-controlled LLM endpoint URLs (67, 188) - External service abuse
• Unrestricted Wikipedia API calls (80) - Proxy attacks
Plugin System Vulnerabilities (High)
• Arbitrary code execution via malicious entry points (61, 128) - Supply chain attacks
• Dynamic plugin loading without security boundaries (62, 129) - Untrusted code execution
• Source adapter code execution via entry point exploitation (128) - Plugin compromise
MCP Server Security Issues (High)
• 29 MCP tools exposed without authentication (99-101) - Unauthorized access to all functionality
• Race condition vulnerabilities in MCP server (923) - Concurrent access issues
• Lack of LLM security boundary (924) - Prompt injection and data exfiltration
Architectural Vulnerabilities (Critical)
• Architectural SQL/NoSQL injection patterns (916) - Systemic query construction flaws
• Architectural graph traversal injection (917) - Knowledge graph manipulation
• Architectural filter injection in search (918) - Search query tampering
• Architectural multi-tenant isolation failure (920) - Systemic data leakage
• Architectural shell injection in hook system (922) - Systemic command execution flaws
ATTACK CHAINS
1. Tenant Data Exfiltration Chain: An attacker can exploit the lack of tenant isolation (4-15, 17-26) combined with the MCP server's lack of authentication (99-101) to access all users' data. The export functionality (12-13) provides a direct exfiltration path.
2. Shell Injection to Full Compromise: The hook system's shell injection vulnerabilities (2-3, 45-54) chain with PID file race conditions (16) and command injection in subprocess calls (89) to achieve arbitrary code execution with the privileges of the MemPalace process.
3. Plugin Supply Chain Attack: The plugin architecture (61-62, 128-130) allows loading untrusted code, which combined with the MCP server's lack of authentication (99-101) creates a remote code execution vector through malicious plugins.
4. SSRF to Internal Network: User-controlled LLM endpoints (67, 188) and ChromaDB paths (60) enable SSRF attacks that can reach internal services, potentially accessing sensitive internal APIs or cloud metadata endpoints.
VERDICT
Immediate Critical Fixes Required:
1. Shell injection in hooks - Replace eval with safe JSON parsing and sanitize all shell command inputs
2. Multi-tenant isolation - Implement tenant context throughout the data layer, especially in backend caching and queries
3. MCP server authentication - Add authentication/authorization to all MCP tools before deployment
4. SQL injection vectors - Parameterize all database queries and validate all user inputs
High Priority:
5. Plugin security - Sandbox plugin execution and validate entry points
6. SSRF protection - Validate and restrict external URL connections
7. Race conditions - Fix PID file handling and concurrent access issues
The codebase has systemic security issues that require architectural changes, particularly around tenant isolation and input validation. The hook system and MCP server are the most exposed attack surfaces.
81 raw scanner findings — 23 critical · 28 high · 29 medium · 1 info
Raw Scanner Output — 855 pre-cleanup findings
⚠ Pre-Cleanup Report
This is the raw, unprocessed output from all scanner agents before AI analysis. Do not use this to fix issues individually. Multiple agents attack from different angles and frequently report the same underlying vulnerability, resulting in significant duplication. Architectural issues also appear as many separate line-level findings when they require a single structural fix.

Use the Copy Fix Workflow button above to get the AI-cleaned workflow — it deduplicates findings, removes false positives, and provides actionable steps. This raw output is provided for transparency and audit purposes only.
HIGHShell hook wrapper enables command injection chain
[redacted]/mempal-stop-hook.sh:1
[AGENTS: Vector]attack_chains
The shell hook wrappers pass stdin directly to Python without validation. An attacker who can control hook input (through compromised AI interactions or other vulnerabilities) could inject shell commands through environment variables or special characters that get interpreted by the shell before being passed to Python. This could be chained with other vulnerabilities for privilege escalation.
Suggested Fix
Use direct Python execution without shell wrappers, or implement strict input validation in the shell scripts.
HIGHCommand injection via hook name parameter
[redacted]/mempal-hook.sh:4
[AGENTS: Infiltrator]attack_surface
The hook script passes the hook name directly to the Python module without proper validation. An attacker could potentially inject command-line arguments or manipulate the hook execution flow by crafting malicious hook names.
Suggested Fix
Validate hook names against a known allowlist, sanitize input before passing to subprocess, and use argument arrays instead of string concatenation.
HIGHShell script with unsafe eval and command injection risk
[redacted]/mempal_save_hook.sh:1
[AGENTS: Tripwire]dependencies
The hook script uses eval on parsed JSON input (line 79-80) which could lead to command injection if the input is maliciously crafted. While there's some sanitization, the pattern is inherently risky.
Suggested Fix
Replace eval with safer parsing using jq or pure Python parsing, avoid shell variable assignment from untrusted input
HIGHShell injection vulnerability in hook script
[redacted]/mempal_save_hook.sh:66
[AGENTS: Gatekeeper]auth
The hook script uses eval to parse JSON input from stdin, which could allow shell injection if the input contains malicious content. While there's some sanitization, the eval approach is inherently risky.
Suggested Fix
Replace eval with a safer parsing method using jq or Python's json module without shell interpolation.
HIGHShell injection vulnerability in hook script
[redacted]/mempal_save_hook.sh:66
[AGENTS: Phantom]api_security
The hook script uses eval on Python output without proper validation. While the Python script attempts to sanitize output, the use of eval with external input creates a potential command injection vector if the Python sanitization fails or is bypassed.
Suggested Fix
Avoid using eval entirely. Use safer parsing methods like jq for JSON parsing or direct Python processing without shell interpolation.
HIGHShell injection via crafted session_id or transcript_path
[redacted]/mempal_save_hook.sh:66
[AGENTS: Infiltrator]attack_surface
The hook script uses eval on Python output without proper sanitization. While there's some sanitization in the Python code, the shell eval creates a potential injection vector if the Python sanitization fails or if there are edge cases in the regex patterns. An attacker could craft malicious session_id or transcript_path values to execute arbitrary commands.
Suggested Fix
Remove eval usage entirely. Use Python to write to a temporary file and source it, or use a more secure method of passing variables between processes.
HIGHShell injection vulnerability in eval statement
[redacted]/mempal_save_hook.sh:66
[AGENTS: Mirage]false_confidence
The script uses eval on Python output without proper validation. While there's a comment about 'Shell-safe output' and a lambda function to sanitize, the actual implementation uses a regex that may not catch all injection vectors. The eval statement creates a false sense of security through partial sanitization.
Suggested Fix
Use a safer parsing method like jq or direct Python-to-shell variable assignment without eval.
HIGHShell injection via Python eval of untrusted JSON
[redacted]/mempal_save_hook.sh:66
[AGENTS: Prompt]llm_security
The hook script uses eval on Python output that parses JSON from stdin. If the JSON contains malicious content that affects the Python parsing, it could lead to shell injection.
Suggested Fix
Use safer parsing: read variables directly from Python without eval: eval $(echo "$INPUT" | python3 -c "import sys, json, re; data = json.load(sys.stdin); print('SESSION_ID=' + repr(data.get('session_id', 'unknown'))); print('STOP_HOOK_ACTIVE=' + repr(str(data.get('stop_hook_active', False)).lower()))")
HIGHShell injection vulnerability in hook state parsing
[redacted]/mempal_save_hook.sh:66
[AGENTS: Compliance]regulatory
The hook script uses eval on Python output without proper input validation, creating potential command injection vulnerabilities. The 'safe' lambda attempts to sanitize but uses a regex that may not catch all dangerous characters. This violates PCI-DSS 6.5.1 (Injection Flaws) and SOC 2 CC6.1 (Logical Access Security) by allowing potentially malicious session data to execute shell commands. The script runs with the same privileges as the Claude Code/Codex CLI process, which could lead to privilege escalation.
Suggested Fix
Replace eval with direct variable assignment using safer methods: 1) Use named pipes or temporary files with restricted permissions, 2) Implement proper shell escaping using printf '%q', 3) Validate all inputs against strict whitelist patterns, 4) Run with reduced privileges where possible.
HIGHUnsafe eval of Python output
[redacted]/mempal_save_hook.sh:66
[AGENTS: Fuse]error_security
The script uses eval on the output of a Python command to set shell variables. While there's sanitization in the Python code, this pattern is inherently risky and could be vulnerable to command injection if the Python code is compromised or modified.
Suggested Fix
Use a safer method to pass data from Python to shell, such as writing to a temporary file or using a more structured IPC method.
HIGHShell injection via eval
[redacted]/mempal_save_hook.sh:66
[AGENTS: Sentinel]input_validation
The script uses eval on Python output: eval $(echo "$INPUT" | python3 -c "..."). If Python output is malicious, it could execute arbitrary shell commands.
Suggested Fix
Use safer parsing: read into variables directly without eval, or use jq for JSON parsing.
HIGHUnsafe eval of untrusted JSON input
[redacted]/mempal_save_hook.sh:83
[AGENTS: Pedant]correctness
The script uses eval on parsed JSON input from stdin without proper sanitization. While there's a Python sanitization step, the eval command still executes arbitrary shell code. If the Python sanitization fails or is bypassed, this could lead to command injection.
Suggested Fix
Replace eval with safer parsing: use jq or a more robust Python parsing that doesn't require eval. Or restructure to avoid shell variable assignment from untrusted input.
HIGHPath traversal in TRANSCRIPT_PATH expansion
[redacted]/mempal_save_hook.sh:106
[AGENTS: Razor]security
The script expands ~ in TRANSCRIPT_PATH but doesn't validate that the resulting path is within expected directories. An attacker could potentially use path traversal sequences like '../../etc/passwd' if they control the transcript_path in the JSON input.
Suggested Fix
Validate TRANSCRIPT_PATH against a whitelist of allowed directories or use realpath to ensure it's within expected bounds.
HIGHSSRF via ChromaDB PersistentClient with malicious palace_path
[redacted]/chroma.py:114
[AGENTS: Specter]ssrf
The _client method creates a ChromaDB PersistentClient with a user-controlled palace_path. While intended to be a local directory path, if an attacker can inject a URL or special path, it might trigger external connections. ChromaDB's PersistentClient might interpret certain paths as URLs.
Suggested Fix
Validate palace_path is a local directory and not a URL before passing to ChromaDB.
HIGHArbitrary code execution via malicious entry points
[redacted]/registry.py:1
[AGENTS: Vector]attack_chains
The backend registry dynamically loads Python classes from entry points without validation. An attacker with write access to Python package directories or control over package repositories could inject malicious code that executes when MemPalace loads backends. This could be chained with package installation vulnerabilities for full system compromise.
Suggested Fix
Implement code signing for entry points, or at least verify that loaded classes conform to expected interfaces before instantiation.
HIGHDynamic plugin loading without security boundaries
[redacted]/registry.py:1
[AGENTS: Infiltrator]attack_surface
The backend registry dynamically loads Python classes from entry points without sandboxing or security validation. Malicious packages could register backends that execute arbitrary code when loaded.
Suggested Fix
Implement a security model for plugins, such as requiring code signing, running plugins in isolated processes, or implementing capability-based security.
HIGHUnsafe LLM endpoint configuration without validation
[redacted]/closet_llm.py:1
[AGENTS: Tripwire]dependencies
The closet_llm module allows arbitrary LLM endpoints via LLM_ENDPOINT environment variable or --endpoint flag. This could lead to: 1) SSRF attacks if endpoint points to internal services, 2) Data exfiltration if endpoint is attacker-controlled, 3) No TLS verification for local endpoints. The code uses urllib without certificate validation.
Suggested Fix
Add endpoint validation (allowlist of trusted domains), enforce HTTPS for non-local endpoints, add certificate verification, and implement request signing for authenticated endpoints.
HIGHLLM integration exposes attack surface for prompt injection and data exfiltration
[redacted]/closet_llm.py:1
[AGENTS: Vector]attack_chains
The closet_llm module allows optional LLM integration for generating richer closets. This exposes multiple attack vectors: 1) Prompt injection through source file content could cause the LLM to execute arbitrary instructions. 2) The LLM endpoint configuration (LLM_ENDPOINT, LLM_KEY) could be hijacked to exfiltrate palace content to attacker-controlled endpoints. 3) Malicious LLM responses could inject harmful content into closets. This creates a multi-step attack chain: compromise LLM endpoint → inject malicious prompts → exfiltrate sensitive data through LLM responses → persist backdoors in closet metadata.
Suggested Fix
Implement strict input sanitization for content sent to LLMs. Validate LLM endpoint URLs against allowlists. Sandbox LLM responses before inserting into closets. Make LLM integration opt-in with explicit warnings about security implications.
HIGHSSRF via user-controlled LLM endpoint URL
[redacted]/closet_llm.py:114
[AGENTS: Specter]ssrf
The `LLMConfig` class accepts an `endpoint` parameter that can be controlled via environment variables or CLI arguments. This endpoint is used to make HTTP requests to an LLM API. An attacker could set the endpoint to an internal service URL (e.g., http://localhost:8080, http://169.254.169.254) to probe internal networks or metadata services, leading to SSRF.
Suggested Fix
Validate the endpoint URL against an allowlist of known safe domains, or implement network-level restrictions to prevent access to internal IP ranges and metadata services.
HIGHDirect LLM prompt injection via user content in closet generation
[redacted]/closet_llm.py:130
[AGENTS: Prompt]llm_security
**Perspective 1:** The `_call_llm()` function in closet_llm.py sends raw user content directly to an LLM API endpoint as part of the prompt without sanitization. The content is embedded in a PROMPT_TEMPLATE that asks the LLM to generate topics, quotes, and summaries. An attacker could craft malicious content containing prompt injection payloads (like 'Ignore previous instructions...') that could hijack the LLM's behavior, potentially causing it to output malicious JSON, leak system prompts, or execute unauthorized actions if the LLM has tool-calling capabilities. The function doesn't validate or sanitize the content before sending it to the LLM. **Perspective 2:** The `_call_llm()` function parses LLM output as JSON without validating the structure against a strict schema. While there's a JSON schema in the prompt, the LLM's response is only checked for basic JSON validity. An attacker could potentially craft content that causes the LLM to output JSON with unexpected fields or malformed values that could lead to downstream issues when processing the closet lines. The function also strips code fences but doesn't validate that the resulting JSON conforms to the expected schema. **Perspective 3:** The `_call_llm()` function sends potentially large content (up to MAX_CONTENT_CHARS = 30000 characters) to an LLM without enforcing token limits. This could lead to excessive API costs or timeouts if malicious or extremely long content is processed. While there's a MAX_OUTPUT_TOKENS limit, there's no input token limit or truncation logic.
Suggested Fix
Implement content filtering to detect and remove common prompt injection patterns, escape special characters in JSON contexts, or use structured prompting techniques that separate user content from instructions more robustly.
HIGHConversation mining registry sentinel enables data poisoning persistence
[redacted]/convo_miner.py:1
[AGENTS: Vector]attack_chains
The _register_file function creates sentinel drawer entries with '[registry]' prefix. Attack chain: 1) Attacker modifies source_file path in sentinel to point to sensitive system file, 2) file_already_mined returns True for that path, 3) Legitimate files with same path are skipped, 4) Data poisoning persists across runs because sentinel remains in database. This creates a persistent denial-of-memory attack where specific files are permanently excluded from mining.
Suggested Fix
Validate source_file paths in registry entries are within allowed directories. Add checksum verification of file content alongside path matching.
HIGHUnvalidated wing parameter in where clause
[redacted]/dedup.py:62
[AGENTS: Syringe]db_injection
The get_source_groups function constructs a where clause using the wing parameter without validation. This parameter flows directly into ChromaDB queries.
Suggested Fix
Validate the wing parameter against known wings or sanitize it before use.
HIGHUnvalidated source_pattern in filter construction
[redacted]/dedup.py:71
[AGENTS: Syringe]db_injection
The get_source_groups function filters by source_pattern using string containment without validation. If source_pattern contains malicious content, it could affect query results.
Suggested Fix
Validate source_pattern parameter or sanitize it before use.
HIGHDiary ingest state file lacks tenant isolation
[redacted]/diary_ingest.py:46
[AGENTS: Tenant]tenant_isolation
The _state_file_for function creates state files keyed only by palace_path and diary_dir hash, with no tenant identifier. This allows Tenant A's ingest state to be overwritten or read by Tenant B if they use the same palace and diary directory paths.
Suggested Fix
Include tenant_id in the state file key generation: hashlib.sha256(f"{tenant_id}|{palace_path}|{diary_dir}".encode()).hexdigest()
HIGHRegex injection vulnerability in entity scoring
[redacted]/entity_detector.py:180
[AGENTS: Sanitizer]sanitization
The _build_patterns function uses re.escape on the entity name but then formats it into regex patterns. If the patterns in language files contain unsafe formatting or the name contains special regex sequences that survive escaping in certain contexts, this could lead to regex injection.
Suggested Fix
Use re.escape on the entire formatted pattern or validate/sanitize patterns in language files during loading.
HIGHUnrestricted Wikipedia API call with user-controlled input
[redacted]/entity_registry.py:185
[AGENTS: Specter]ssrf
The `_wikipedia_lookup` function makes HTTP requests to Wikipedia API with user-controlled `word` parameter without validation or restriction. This could allow SSRF attacks where an attacker could use the application as a proxy to internal services or perform port scanning.
Suggested Fix
Implement allowlist validation for words before making external requests, add rate limiting, and restrict allowed domains. Consider using a local dictionary instead of external API calls.
HIGHUnsanitized input in Wikipedia lookup function
[redacted]/entity_registry.py:189
[AGENTS: Phantom]api_security
**Perspective 1:** The _wikipedia_lookup function directly uses user-provided word in URL construction without proper sanitization. While urllib.parse.quote is used, the function makes external HTTP requests with user-controlled data, potentially exposing internal network information or enabling SSRF attacks if the function is called with malicious input. **Perspective 2:** The Wikipedia lookup function creates a dependency on an external API (en.wikipedia.org) without authentication, rate limiting, or fallback mechanisms. This could lead to service disruption if Wikipedia blocks the requests or if the function is called excessively.
Suggested Fix
Validate input before making external requests, implement allowlists for characters, and add timeout/retry limits. Consider implementing a local cache to minimize external requests.
HIGHSQL injection risk in knowledge graph queries
[redacted]/fact_checker.py:129
[AGENTS: Syringe]db_injection
The code queries the knowledge graph with user-provided entity names without proper parameterization: 'facts = kg.query_entity(subject, direction="outgoing")'. The subject parameter comes from text extraction and could contain malicious content if the underlying query implementation doesn't use parameterized queries.
Suggested Fix
Ensure the knowledge graph query methods use parameterized queries or properly escape input values.
HIGHHook system processes untrusted JSON from stdin
[redacted]/hooks_cli.py:1
[AGENTS: Infiltrator]attack_surface
The hooks_cli.py reads JSON from stdin and executes hook logic based on the input. It validates session_id and transcript_path but an attacker could potentially craft malicious JSON to trigger unexpected behavior, especially if the hook is invoked from an untrusted process.
Suggested Fix
Add HMAC signature validation for hook inputs, or restrict hook invocation to trusted parent processes only.
HIGHPID file race condition and stale PID handling
[redacted]/hooks_cli.py:1
[AGENTS: Chaos]edge_cases
**Perspective 1:** _spawn_mine() writes PID to a file, but there's a race condition between checking _mine_already_running() and writing the new PID. Also, stale PID files from crashed processes could prevent new mining operations indefinitely. **Perspective 2:** _mine_sync() runs subprocess with a 60-second timeout, but _spawn_mine() (background mining) has no timeout. A hanging mine process could run indefinitely, consuming resources and blocking future hooks.
Suggested Fix
Use file locking for PID file operations. Implement stale PID cleanup by checking process existence and last modification time.
HIGHCommand injection in subprocess calls
[redacted]/hooks_cli.py:213
[AGENTS: Sanitizer]sanitization
The _spawn_mine and _mine_sync functions use subprocess.Popen/subprocess.run with command lists constructed from user-influenced paths (mine_dir from transcript_path). While the paths are validated, there's still risk if validation is bypassed.
Suggested Fix
Use shlex.quote on path arguments or additional validation before passing to subprocess.
HIGHUnvalidated wing parameter in where clause
[redacted]/layers.py:114
[AGENTS: Syringe]db_injection
The Layer1.generate method constructs a where clause using the wing parameter without validation. This parameter flows directly into ChromaDB queries.
Suggested Fix
Validate the wing parameter against known wings or sanitize it before use.
HIGHUnvalidated wing and room parameters in where clause
[redacted]/layers.py:200
[AGENTS: Syringe]db_injection
The Layer2.retrieve method uses user-provided wing and room parameters to construct a where filter without validation. These parameters flow directly into ChromaDB queries.
Suggested Fix
Implement input validation for wing and room parameters before constructing the where clause.
HIGHUnvalidated wing and room parameters in search query
[redacted]/layers.py:244
[AGENTS: Syringe]db_injection
The Layer3.search method uses user-provided wing and room parameters to construct a where filter without validation. These parameters flow directly into ChromaDB queries.
Suggested Fix
Validate wing and room parameters before constructing the where clause.
HIGHUnvalidated wing and room parameters in search_raw query
[redacted]/layers.py:287
[AGENTS: Syringe]db_injection
The Layer3.search_raw method uses user-provided wing and room parameters to construct a where filter without validation. These parameters flow directly into ChromaDB queries.
Suggested Fix
Validate wing and room parameters before constructing the where clause.
HIGHMCP server exposes 29 tools without authentication
[redacted]/mcp_server.py:0
[AGENTS: Phantom]api_security
**Perspective 1:** The MCP server exposes 29 different tools (mempalace_status, mempalace_list_wings, mempalace_search, mempalace_add_drawer, mempalace_delete_drawer, etc.) without any authentication mechanism. Any client connecting via stdio can execute all operations including reading all stored data, adding/updating/deleting drawers, and modifying the knowledge graph. This is a classic Broken Function Level Authorization (BFL) vulnerability where all functions are accessible without authentication. **Perspective 2:** The MCP server exposes write operations (mempalace_add_drawer, mempalace_delete_drawer, mempalace_update_drawer, mempalace_kg_add, mempalace_kg_invalidate, etc.) without any rate limiting. An attacker could flood the system with write requests, potentially causing denial of service, data corruption, or exhausting storage resources. **Perspective 3:** The write-ahead log (WAL) system logs all write operations to ~/.mempalace/wal/write_log.jsonl, but only redacts specific keys (content, content_preview, document, entry, entry_preview, query, text). Other potentially sensitive metadata fields are logged in plaintext. Additionally, the WAL file permissions (0o600) are set but there's no guarantee they're respected on all filesystems, and the log accumulates indefinitely without rotation or encryption. **Perspective 4:** All tools provide unrestricted access to all data in the palace. There's no concept of ownership, access control, or data segregation. Once connected, a client can read all drawers, search all content, and access all knowledge graph entries regardless of sensitivity. **Perspective 5:** While some tools use sanitize_name() and sanitize_content(), others like tool_delete_tunnel() accept tunnel_id without validation. The tool_delete_drawer() accepts drawer_id without validation beyond checking existence. This could lead to injection attacks if these IDs are used in database queries or filesystem operations. **Perspective 6:** The server may expose internal error details through exception messages that could reveal system information, file paths, or database structure to clients. **Perspective 7:** The MCP server provides 29 tools for reading and writing to the memory palace. While this is by design for local AI integration, it creates a large attack surface if the server is exposed to untrusted networks. No authentication mechanism is mentioned in the provided code.
Suggested Fix
Implement authentication for the MCP server, such as requiring an API key or token that must be validated before processing any tool calls. Consider implementing role-based access control for different operations.
HIGH29 MCP tools exposed without authentication
[redacted]/mcp_server.py:0
[AGENTS: Infiltrator]attack_surface
The MCP server exposes 29 tools for reading and writing to the palace without any authentication mechanism. While MCP typically runs locally, if the server is accidentally exposed or runs in a network-accessible configuration, it provides full access to memory storage and knowledge graph operations. No rate limiting or access controls are implemented.
Suggested Fix
Implement authentication for network-exposed configurations, add rate limiting, and consider implementing role-based access control for different tool categories.
HIGHMCP Server exposes 29+ tools without authentication
[redacted]/mcp_server.py:1
[AGENTS: Infiltrator]attack_surface
The MCP server exposes 29+ tools (mempalace_status, mempalace_search, mempalace_add_drawer, mempalace_delete_drawer, etc.) via JSON-RPC over stdio with no authentication mechanism. Any process that can invoke this server gains full read/write access to the palace database, including adding/updating/deleting drawers, querying the knowledge graph, and writing diary entries. The server runs with the user's privileges and can modify the palace database directly.
Suggested Fix
Add authentication token validation for MCP connections, or restrict server to trusted processes only via process-level isolation.
HIGHMCP server exposes write-ahead log with sensitive content redaction bypass
[redacted]/mcp_server.py:1
[AGENTS: Vector]attack_chains
**Perspective 1:** The MCP server implements a write-ahead log (WAL) for audit trails but uses a limited set of redaction keys (_WAL_REDACT_KEYS). Attackers can chain this with content injection to bypass redaction: 1) Inject content with non-standard field names not in the redaction list, 2) Use the MCP tools to add drawers with sensitive data in non-redacted fields, 3) The WAL file (~/.mempalace/wal/write_log.jsonl) persists unredacted sensitive data. This creates a data exfiltration path where attackers can embed credentials, tokens, or PII in metadata fields that won't be redacted, then extract them from the WAL file. **Perspective 2:** The WAL file creation has a TOCTOU race: 1) File is created with os.open(O_CREAT|O_WRONLY, 0o600), 2) If attacker can replace the file between creation and chmod (on systems where os.open doesn't atomically set permissions), they can create a symlink to sensitive files. Chained with: A) Attacker gains low-privilege access to run MCP server, B) Uses timing attack to replace WAL file with symlink to /etc/passwd or other sensitive file, C) MCP server writes audit logs, overwriting target file. This enables privilege escalation through file corruption or controlled overwrites of sensitive system files. **Perspective 3:** The _extract_entities_for_metadata function extracts entity names from content using regex patterns and a known-entity registry. Attack chain: 1) Attacker injects malicious entity names into source files (e.g., 'DROP TABLE users;' as an entity), 2) Miner processes files, extracts entities, stores in metadata, 3) Downstream ML systems using these embeddings for training ingest poisoned data, 4) Model poisoning leads to degraded performance or backdoors. This is a supply chain attack vector where poisoned training data affects all users of models trained on MemPalace outputs.
Suggested Fix
Implement a deny-list approach instead of allow-list: redact ALL values except explicitly safe fields (like 'wing', 'room', 'drawer_id'). Or better, hash all content values in WAL entries rather than storing any raw content.
HIGHSQL injection risk in raw SQL queries
[redacted]/migrate.py:1
[AGENTS: Chaos]edge_cases
**Perspective 1:** The extract_drawers_from_sqlite function uses string formatting in SQL queries without parameterization for the embedding_id value. **Perspective 2:** If another process has the SQLite database open (e.g., another mempalace instance), the migration will fail or cause corruption.
Suggested Fix
Check for database locks and provide clear error message, or implement retry logic.
HIGHDatabase migration with direct SQLite access
[redacted]/migrate.py:1
[AGENTS: Infiltrator]attack_surface
The migrate module reads directly from SQLite databases and performs schema detection and data extraction. This creates a potential attack surface where maliciously crafted SQLite files could exploit SQLite vulnerabilities or cause denial of service.
Suggested Fix
Add input validation for SQLite files, limit maximum file size, and consider using read-only mode for extraction operations.
HIGHRaw SQL query construction without parameterization
[redacted]/migrate.py:62
[AGENTS: Syringe]db_injection
The 'extract_drawers_from_sqlite' function constructs SQL queries using string concatenation with potentially untrusted input. While this is for migration and uses hardcoded queries, the pattern is dangerous: 'WHERE e.embedding_id = ?' uses parameterization, but other parts of the query use string operations.
Suggested Fix
Use parameterized queries throughout and avoid string concatenation for SQL fragments.
HIGHDynamic SQL schema inspection without validation
[redacted]/migrate.py:114
[AGENTS: Syringe]db_injection
The 'detect_chromadb_version' function executes raw SQL to inspect database schema: 'PRAGMA table_info(collections)'. While this is for migration detection, executing dynamic SQL based on external input is risky.
Suggested Fix
Use database metadata APIs instead of raw SQL when possible, or validate all inputs.
HIGHSQL injection risk in direct SQLite queries
[redacted]/migrate.py:248
[AGENTS: Razor]security
The `extract_drawers_from_sqlite` function uses string concatenation in SQL queries instead of parameterized queries. While the input is a file path, the function executes raw SQL that could be vulnerable if the database is maliciously crafted.
Suggested Fix
Use parameterized queries for all SQL operations, even with trusted inputs, to prevent SQL injection.
HIGHEntity registry loads external JSON file with user-defined entities
[redacted]/miner.py:1
[AGENTS: Egress]data_exfiltration
**Perspective 1:** The miner module loads entity names from ~/.mempalace/known_entities.json and uses them to tag content metadata. This external JSON file could contain sensitive entity names (people, projects) that get embedded in ChromaDB metadata. The entities are then exposed through the metadata field and could be queried or leaked through search results. There's no validation or sanitization of the entity names loaded from this external file. **Perspective 2:** The mine() function in miner.py prints detailed progress information to stdout including file names being processed ('+ [{i:4}/{len(files)}] {filepath.name[:50]:50} +{drawers}'). This could leak sensitive information about file system structure and content being indexed. In headless or automated environments where stdout is captured, this creates a data exfiltration vector.
Suggested Fix
1. Add validation for entity names loaded from external files. 2. Consider encrypting the entity registry file. 3. Add user consent/confirmation before loading external entity definitions.
HIGHFile system traversal during mining without proper sandboxing
[redacted]/miner.py:1
[AGENTS: Infiltrator]attack_surface
The miner.py scans project directories, reads files up to 500MB, and processes them into the palace. It follows symlinks (though it checks for them) and processes files based on extensions. An attacker could potentially cause denial of service by pointing it at /dev/urandom or other large files, or exfiltrate data via specially crafted filenames that get embedded in metadata.
Suggested Fix
Implement resource limits (CPU, memory, file descriptors), sandbox mining operations, validate file paths more strictly.
HIGHRace condition in mine_lock with concurrent miners
[redacted]/miner.py:1
[AGENTS: Chaos]edge_cases
**Perspective 1:** The mine_lock() context manager (imported from .palace) is used to lock files during processing, but if two miners run simultaneously on the same project directory, they could still interleave operations on different files, potentially causing inconsistent state in the ChromaDB collection. **Perspective 2:** The process_file() function reads files with UTF-8 encoding and errors='replace', but doesn't handle cases where replacement characters could break semantic meaning or cause embedding issues. Files with mixed encodings or binary data masquerading as text could produce garbage content that's still filed. **Perspective 3:** If mining fails partway through (e.g., due to disk full, permission error, or crash), partially filed drawers remain in the palace without rollback. Subsequent mining runs might skip these files (if file_already_mined returns True) or create duplicates. **Perspective 4:** _extract_entities_for_metadata() extracts entity names and joins them with semicolons. With very long content containing many entities, this could exceed ChromaDB metadata size limits or cause performance issues during filtering.
Suggested Fix
Implement transactional mining: stage changes in a temporary collection, then atomically commit. Or implement a cleanup mechanism to remove orphaned drawers from failed mining sessions.
HIGHGitignore bypass via symlink traversal enables arbitrary file read
[redacted]/miner.py:1
[AGENTS: Vector]attack_chains
**Perspective 1:** The gitignore matching logic checks symlinks but doesn't resolve them properly. Attack chain: 1) Attacker creates symlink in project directory pointing to /etc/passwd, 2) Symlink has .txt extension (or other readable extension), 3) is_symlink() check passes, but the symlink target is outside project_path, 4) relative_to(project_path) fails with ValueError, causing the function to return None, 5) is_gitignored returns False (decision is None), 6) File gets processed and content exfiltrated. This bypasses both gitignore and path traversal protections. **Perspective 2:** The _ENTITY_REGISTRY_CACHE is shared across all miner instances and loaded from ~/.mempalace/known_entities.json. Attack chain: 1) Attacker with write access to home directory modifies known_entities.json, 2) Adds malicious entity patterns (e.g., regex injection patterns), 3) Next file processing uses poisoned cache, 4) Entity extraction produces unexpected results, potentially bypassing content filters or injecting malicious metadata. This is a persistence mechanism: once the cache is poisoned, all subsequent mining operations are affected until cache is cleared.
Suggested Fix
When encountering symlinks, resolve them with Path.resolve() and check if resolved path is still within project_path. Reject any symlink whose target is outside the project directory.
HIGHPrivilege escalation through palace repair functionality
[redacted]/repair.py:1
[AGENTS: Vector]attack_chains
The repair functionality can delete and recreate palace collections. An attacker with write access to the palace directory could chain this with other vulnerabilities to escalate privileges by corrupting the palace database, then using the repair function to restore it with malicious content, potentially gaining control over the memory system.
Suggested Fix
Add strict permission checks and user confirmation for destructive operations, and implement integrity verification before repair.
HIGHDestructive file operations without sufficient validation
[redacted]/repair.py:1
[AGENTS: Infiltrator]attack_surface
The repair module performs destructive operations like deleting collections, backing up SQLite databases, and moving directories. While it requires confirmation, the operations affect the entire palace database and could be triggered accidentally or maliciously.
Suggested Fix
Add additional safeguards like requiring a specific flag for destructive operations, implementing dry-run by default, and creating backup verification before deletion.
HIGHSource adapter code execution via entry point exploitation
[redacted]/registry.py:1
[AGENTS: Vector]attack_chains
Similar to the backend registry, the source adapter registry loads arbitrary Python code from entry points. This creates a supply chain attack vector where compromised source adapter packages could execute arbitrary code in the MemPalace process, leading to full system compromise when combined with package management vulnerabilities.
Suggested Fix
Add sandboxing for source adapters, validate loaded code, and implement permission boundaries for adapter operations.
HIGHUnrestricted source adapter loading
[redacted]/registry.py:1
[AGENTS: Infiltrator]attack_surface
Similar to the backend registry, the source adapter registry loads arbitrary Python classes from entry points. This creates a large attack surface as source adapters handle file I/O and could be compromised.
Suggested Fix
Implement adapter sandboxing, require explicit user approval for third-party adapters, and audit adapter capabilities.
HIGHTransformation protocol enables plugin injection attacks
[redacted]/transforms.py:1
[AGENTS: Vector]attack_chains
The Transformation protocol and RESERVED_TRANSFORMATIONS registry allow adapters to register custom transformations. Attack chain: 1) Attacker creates malicious adapter with custom transformation, 2) Transformation executes arbitrary code during content processing, 3) Since transformations are applied in pipeline order, attacker can intercept and modify all content. This is a plugin system vulnerability where third-party adapters have unrestricted access to content processing pipeline.
Suggested Fix
Implement sandboxing for transformation execution. Use a restricted subset of Python (e.g., ast.literal_eval only). Validate adapter signatures before loading.
HIGHSweeper cursor tracking lacks tenant isolation
[redacted]/sweeper.py:172
[AGENTS: Tenant]tenant_isolation
The get_palace_cursor function queries the collection with where={"session_id": session_id} but no tenant_id filter. This allows Tenant A to see cursor positions for Tenant B's sessions if they share the same session_id pattern. The sweep function also uses the same collection without tenant scoping.
Suggested Fix
Add tenant_id to metadata and include it in all WHERE clauses for cursor tracking and sweep operations.
HIGHEmail collection to third-party endpoint without privacy notice
[redacted]/useLandingEffects.js:13
[AGENTS: Egress]data_exfiltration
The waitlist form submits email addresses to 'https://br.staging.mempalaceofficial.com/waitlist' without clear privacy policy disclosure or data handling terms. This endpoint is on a different domain (staging subdomain) and could be collecting PII without proper consent mechanisms.
Suggested Fix
Add explicit privacy notice, terms of service acceptance, and data handling information before submission. Consider using a self-hosted endpoint or clearly disclosing third-party data processing.
HIGHEmail collection without privacy policy or data handling disclosure
[redacted]/useLandingEffects.js:13
[AGENTS: Warden]privacy
**Perspective 1:** The waitlist form sends email addresses to 'https://br.staging.mempalaceofficial.com/waitlist' without displaying a privacy policy, data retention terms, or information about how the data will be used. This violates GDPR and other privacy regulations requiring transparency about data processing. **Perspective 2:** The waitlist form collects email addresses for updates without obtaining explicit consent for marketing communications or providing an opt-out mechanism.
Suggested Fix
Add privacy policy link, data retention information, and consent checkbox before form submission. Clearly state how email addresses will be used.
HIGH[Architectural] Architectural lack of input validation contract (17 instances)
[redacted]/dialect.py:0
[AGENTS: architectural-scanner]architectural
ROOT CAUSE: Missing centralized input validation layer for all public API methods 17 instances of missing validation across different methods indicate no architectural contract for input validation. Each method independently handles (or ignores) validation of parameters like entity names, file paths, JSON inputs, and configuration values. This violates the principle of defensive programming and creates inconsistent security posture. This architectural issue produced 17 individual findings that cannot be resolved with line-by-line patches.
Suggested Fix
Create a centralized validation module with decorators or a validation service. All public methods in dialect.py should be decorated with @validate_input(schema) where schema defines constraints for each parameter. The validation layer should handle: 1) type checking, 2) length limits, 3) safe character sets, 4) path traversal prevention, 5) null byte rejection. This eliminates the need for per-method validation logic.
HIGH[Architectural] Missing validation gateway for external data sources (9 instances)
[redacted]/entity_registry.py:0
[AGENTS: architectural-scanner]architectural
ROOT CAUSE: No validation layer for external API interactions and user inputs 9 instances show missing validation for Wikipedia API responses, user inputs, JSON parsing, and URL construction. This is an architectural issue because external data sources require consistent validation, sanitization, and size limiting. Individual patches won't address the systemic lack of validation boundaries between trusted and untrusted data. This architectural issue produced 9 individual findings that cannot be resolved with line-by-line patches.
Suggested Fix
Implement an ExternalDataGateway class that wraps all external API calls (Wikipedia, HTTP requests). This gateway should: 1) validate and sanitize all inputs before constructing requests, 2) enforce response size limits, 3) validate JSON structure with schemas, 4) implement timeout and retry policies. All external calls in entity_registry.py should route through this gateway.
HIGH[Architectural] Architectural race condition vulnerabilities (5 instances)
[redacted]/mcp_server.py:0
[AGENTS: architectural-scanner]architectural
ROOT CAUSE: Race conditions and TOCTOU vulnerabilities in file operations 5 instances show race conditions in WAL file permissions, diary ID generation, and file operations. This is architectural because the code doesn't use atomic filesystem operations or proper locking mechanisms, creating security vulnerabilities through time-of-check-time-of-use flaws. This architectural issue produced 5 individual findings that cannot be resolved with line-by-line patches.
Suggested Fix
Implement a AtomicFileOperations utility with: 1) Atomic file creation with O_EXCL, 2) Proper file locking (fcntl/portalocker), 3) Secure temporary file patterns, 4) Atomic rename operations. All file operations in mcp_server.py should use this utility instead of direct os/open calls.
HIGH[Architectural] Architectural lack of LLM security boundary (4 instances)
[redacted]/mcp_server.py:0
[AGENTS: architectural-scanner]architectural
ROOT CAUSE: Missing LLM prompt injection protection layer 4 instances of LLM prompt injection via user-controlled content indicate no architectural protection against prompt injection attacks. The system passes user content directly to LLM prompts without sanitization or validation. This architectural issue produced 4 individual findings that cannot be resolved with line-by-line patches.
Suggested Fix
Implement a PromptSanitizer class that: 1) Detects and blocks common injection patterns, 2) Validates content before inclusion in prompts, 3) Implements output filtering for sensitive data, 4) Uses allowlists for predicate values. All LLM interactions should route through this sanitizer.
MEDIUMMissing artifact signing for plugin distribution
[redacted]/marketplace.json:1
[AGENTS: Supply]supply_chain
The plugin marketplace configuration does not specify artifact signing or verification for the mempalace plugin distribution.
Suggested Fix
Add digital signature verification for plugins and require signed artifacts in the marketplace policy.
MEDIUMShell script hook executes arbitrary Python code without validation
[redacted]/mempal-precompact-hook.sh:1
[AGENTS: Harbor]containers
Similar to the stop hook, this script reads raw stdin and pipes it to Python without validation. In a containerized environment, hooks should validate their inputs to prevent command injection or unexpected behavior.
Suggested Fix
Implement input validation, size limits, and structured data parsing before passing to the Python CLI.
MEDIUMShell script executes arbitrary input without validation
[redacted]/mempal-precompact-hook.sh:1
[AGENTS: Lockdown]configuration
Same issue as the stop hook - the precompact hook reads JSON from stdin and passes it directly to Python without validation, creating a potential injection vector.
Suggested Fix
Implement the hook entirely in Python or add input validation at the shell script level.
MEDIUMHook scripts process conversation transcripts without encryption
[redacted]/mempal-stop-hook.sh:1
[AGENTS: Warden]privacy
The hook scripts process conversation transcripts (via stdin) which may contain sensitive personal data, but the data is passed through shell scripts without encryption in transit.
Suggested Fix
Implement encryption for hook data transmission or ensure hooks run in secure, isolated environments.
MEDIUMShell script hook executes arbitrary Python code without validation
[redacted]/mempal-stop-hook.sh:1
[AGENTS: Harbor]containers
The hook script reads raw input from stdin and pipes it directly to Python without any input validation or sanitization. This could allow injection attacks if the hook is called from an untrusted context or if the input contains malicious content.
Suggested Fix
Add input validation, limit input size, and consider using a structured format like JSON with schema validation before passing to Python.
MEDIUMShell script executes arbitrary input without validation
[redacted]/mempal-stop-hook.sh:1
[AGENTS: Lockdown]configuration
The hook script reads JSON from stdin and passes it directly to a Python script without validation. While the Python script handles the parsing, this creates an unnecessary shell layer that could be exploited if the Python script has vulnerabilities.
Suggested Fix
Consider implementing the hook entirely in Python to avoid shell injection risks, or at least validate the input format before passing it along.
MEDIUMNo error handling for Python module import failure
[redacted]/mempal-stop-hook.sh:1
[AGENTS: Chaos]edge_cases
**Perspective 1:** The shell script pipes input to Python without checking if the mempalace module is installed or importable. If Python fails, the hook fails silently. **Perspective 2:** While the input is piped, if the script were modified to use the input differently, there could be injection risks. Currently safe but pattern is risky.
Suggested Fix
Explicitly use piping as done, avoid eval or command substitution with user input.
MEDIUMHook scripts without security controls or audit logging
[redacted]/mempal-stop-hook.sh:1
[AGENTS: Compliance]regulatory
Hook scripts execute automatically without security controls, input validation, or audit logging. This violates SOC 2 CC6.1 (Logical Access Security) and PCI-DSS 6.5 (Address common coding vulnerabilities) requirements for secure script execution.
Suggested Fix
Add input validation, security controls, and audit logging to hook scripts.
MEDIUMShell script injection via environment variables
[redacted]/mempal-stop-hook.sh:4
[AGENTS: Infiltrator]attack_surface
The hook scripts use bash with environment variable expansion (${CLAUDE_PLUGIN_ROOT}) without proper validation. If CLAUDE_PLUGIN_ROOT can be controlled by an attacker, it could lead to command injection.
Suggested Fix
Validate CLAUDE_PLUGIN_ROOT path before use, or use Python-based hooks exclusively to avoid shell injection risks.
MEDIUMShell command injection via pipe input
[redacted]/mempal-stop-hook.sh:6
[AGENTS: Razor]security
The hook script reads stdin and pipes it directly to a Python command without validation. If an attacker can control the input, they could inject shell commands through special characters or newlines.
Suggested Fix
Use `printf '%s' "$INPUT"` or similar safe methods to pass data, or validate input doesn't contain shell metacharacters.
MEDIUMMissing plugin integrity verification
[redacted]/plugin.json:1
[AGENTS: Supply]supply_chain
The Claude plugin configuration doesn't include integrity checks (checksums/signatures) for the plugin files, allowing tampering.
Suggested Fix
Add SHA256 checksums to plugin.json or implement plugin signing.
MEDIUMPotential command injection in plugin hook system
[redacted]/mempal-hook.sh:1
[AGENTS: Phantom]api_security
The hook script passes user-provided hook names directly to the Python module without validation. If an attacker can control the hook name parameter, they could potentially inject command arguments.
Suggested Fix
Validate hook names against a known allowlist before passing to the Python module.
MEDIUMTemporary file creation without secure permissions
[redacted]/mempal-hook.sh:4
[AGENTS: Specter]injection
The script creates a temporary file with `mktemp` but doesn't set secure permissions, potentially allowing other users to read sensitive hook input data.
Suggested Fix
Use `mktemp` with a more restrictive umask or create the file in a user-specific directory with proper permissions.
MEDIUMInsecure temporary file creation
[redacted]/mempal-hook.sh:4
[AGENTS: Razor]security
The script creates a temporary file with mktemp but doesn't set restrictive permissions or ensure it's created in a secure directory. An attacker could potentially race to read or modify the file.
Suggested Fix
Use mktemp with more secure options: INPUT_FILE=$(mktemp -t mempal-hook.XXXXXX) && chmod 600 "$INPUT_FILE"
MEDIUMMissing plugin integrity verification
[redacted]/plugin.json:1
[AGENTS: Supply]supply_chain
Codex plugin configuration doesn't include integrity checks or signature verification for the MCP server command. Malicious plugins could substitute different binaries.
Suggested Fix
Add checksum verification for the python binary path or implement plugin signing for Codex CLI integration.
MEDIUMIncomplete security incident response documentation
[redacted]/SECURITY.md:34
[AGENTS: Compliance]regulatory
The security policy mentions acknowledgment within 48 hours but lacks detailed incident response procedures required by SOC 2 CC7.3 and PCI-DSS 12.10.
Suggested Fix
Expand security policy to include incident response procedures, escalation paths, and communication plans.
MEDIUMBenchmark documentation presents potentially misleading comparison table
[redacted]/BENCHMARKS.md:1
[AGENTS: Mirage]false_confidence
The BENCHMARKS.md file includes a comparison table showing MemPalace vs other systems with R@5 scores, but includes a caveat that some systems report different metrics (QA accuracy vs retrieval recall). The documentation states 'Public-facing pages no longer present this table' due to issue #875, suggesting awareness of the misleading nature, yet the table remains in the documentation creating potential false confidence for readers who don't read the full caveat.
Suggested Fix
Remove the comparison table or restructure it to only compare like metrics with clear labeling.
MEDIUMOverconfident benchmark claims without reproducible setup
[redacted]/BENCHMARKS.md:1
[AGENTS: Provenance]ai_provenance
The document makes strong claims about performance (96.6% to 100% recall) but includes caveats about methodological issues like 'teaching to the test' and contaminated dev sets. The instructions for reproducing results assume specific data files and API keys without verification steps.
Suggested Fix
Add verification steps and clearer warnings about methodological limitations, with explicit instructions for clean reproduction.
MEDIUMComprehensive benchmark results expose system capabilities and limitations
[redacted]/BENCHMARKS.md:1
[AGENTS: Recon]info_disclosure
This markdown file contains exhaustive benchmark results across multiple datasets (LongMemEval, LoCoMo, ConvoMem, MemBench) with detailed performance breakdowns by question type, retrieval modes, and comparison with competitors. It reveals specific weaknesses (e.g., 'noisy' category at 43.4%, 'preference' category at 86.0%), architectural decisions, and implementation details that could help attackers understand system limitations and craft adversarial inputs.
Suggested Fix
Move detailed benchmark results to internal documentation. Public documentation should focus on high-level capabilities without exposing specific performance characteristics and weaknesses.
MEDIUMDetailed architecture documentation exposes system internals
[redacted]/HYBRID_MODE.md:1
[AGENTS: Recon]info_disclosure
The HYBRID_MODE.md file (551 lines) provides a comprehensive technical writeup of the hybrid retrieval system, including exact formulas, weight values, failure analysis, and implementation details. This gives attackers deep insight into the system's design decisions and potential attack vectors.
Suggested Fix
Move detailed technical documentation to internal repositories. Provide only high-level overviews in public documentation.
MEDIUMOverconfident performance claims without reproducibility details
[redacted]/HYBRID_MODE.md:1
[AGENTS: Provenance]ai_provenance
The document makes specific performance claims (R@5 percentages) but lacks exact reproduction commands for all modes. Some results reference 'palace mode' without clear definition of what palace mode entails.
Suggested Fix
Add exact reproduction commands for each mode or reference to a reproducible script with seed values.
MEDIUMOverconfident benchmark claims without reproducible setup verification
[redacted]/README.md:1
[AGENTS: Provenance]ai_provenance
The README makes specific performance claims (96.6%, 84.2%, 89.4% results) and provides detailed reproduction instructions, but there's no verification that the benchmark setup actually produces these results. The instructions assume specific data availability and environment setup without validation.
Suggested Fix
Include validation steps in the benchmark scripts to verify data availability and environment compatibility.
MEDIUMBenchmark documentation presents potentially misleading results
[redacted]/README.md:1
[AGENTS: Mirage]false_confidence
The README presents benchmark results (96.6%, 84.2%, 89.4%) without context about limitations or potential overfitting. It claims 'No API key. No internet during benchmark' but doesn't address potential data leakage or benchmark-specific optimizations that don't generalize.
Suggested Fix
Include limitations section and caveats about real-world performance vs. benchmark performance.
MEDIUMMissing integrity verification for downloaded benchmark data
[redacted]/convomem_bench.py:1
[AGENTS: Supply]supply_chain
The script downloads ConvoMem benchmark data from HuggingFace without verifying checksums or signatures, allowing data tampering.
Suggested Fix
Add SHA256 checksum verification for downloaded JSON files.
MEDIUMUnpinned ChromaDB dependency in third benchmark script
[redacted]/convomem_bench.py:1
[AGENTS: Tripwire]dependencies
Third benchmark script with unpinned chromadb dependency, increasing attack surface and compatibility risks.
Suggested Fix
Add unified dependency management for all benchmark scripts
MEDIUMConvoMem benchmark exposes evidence matching and retrieval strategies
[redacted]/convomem_bench.py:1
[AGENTS: Recon]info_disclosure
This benchmark script reveals how the system matches evidence messages, handles different evidence categories, and performs retrieval. It includes specific HuggingFace dataset URLs and caching strategies that could help attackers understand data sources and evaluation methodologies.
Suggested Fix
Abstract dataset URLs and evidence matching logic to reduce information exposure.
MEDIUMSSRF via HuggingFace URL construction
[redacted]/convomem_bench.py:65
[AGENTS: Specter]ssrf
The `download_evidence_file` function constructs URLs by concatenating user-controlled `category` and `subpath` parameters without validation. An attacker could potentially craft malicious paths to access unintended resources.
Suggested Fix
Validate category and subpath parameters against expected patterns and restrict to alphanumeric characters and safe separators.
MEDIUMOutbound HTTP request to HuggingFace for dataset download
[redacted]/convomem_bench.py:66
[AGENTS: Egress]data_exfiltration
The download_evidence_file function downloads benchmark data from HuggingFace (https://huggingface.co/datasets/Salesforce/ConvoMem/resolve/main/...). While this is expected for benchmarking, it represents an outbound data flow that could be monitored or intercepted, and the dataset contains conversational data that may include sensitive examples.
Suggested Fix
Document the external data dependency and consider local caching with integrity verification.
MEDIUMBroad exception catching in file download
[redacted]/convomem_bench.py:78
[AGENTS: Fuse]error_security
The download_evidence_file function catches generic Exception when downloading from HuggingFace, then prints a generic error message. This hides network errors, authentication issues, or file corruption.
Suggested Fix
Catch specific exceptions (urllib.error.URLError, urllib.error.HTTPError, socket.timeout) and provide appropriate error messages for each case.
MEDIUMSSRF via HuggingFace API discovery endpoint
[redacted]/convomem_bench.py:84
[AGENTS: Specter]ssrf
The `discover_files` function makes requests to HuggingFace's API endpoint with user-controlled `category` parameter in the URL path. While limited to HuggingFace domains, this could still be abused to probe for different resources.
Suggested Fix
Validate category parameter against a known list of valid ConvoMem categories.
MEDIUMOutbound HTTP request to HuggingFace API for file discovery
[redacted]/convomem_bench.py:84
[AGENTS: Egress]data_exfiltration
The discover_files function makes API calls to HuggingFace (https://huggingface.co/api/datasets/Salesforce/ConvoMem/tree/main/...) to discover available files. This creates an outbound connection that could leak metadata about the benchmarking activity.
Suggested Fix
Cache file listings locally to reduce external calls and document the dependency.
MEDIUMBroad exception catching in file discovery
[redacted]/convomem_bench.py:93
[AGENTS: Fuse]error_security
The discover_files function catches generic Exception when querying the HuggingFace API, then prints a generic error message. This could hide network issues, API changes, or authentication problems.
Suggested Fix
Catch specific exceptions and log appropriate error messages. Consider implementing a fallback mechanism when API discovery fails.
MEDIUMUncontrolled HTTP download from external source
[redacted]/convomem_bench.py:343
[AGENTS: Razor]security
The `download_evidence_file` function downloads files from HuggingFace URLs without validating the URL or content. While it's a known domain, the function could be abused if URL parameters are manipulated.
Suggested Fix
Validate downloaded content size and type, implement checksum verification, and restrict downloads to expected file types.
MEDIUMMissing error handling in download_evidence_file
[redacted]/convomem_bench.py:343
[AGENTS: Pedant]correctness
**Perspective 1:** The download_evidence_file function doesn't handle cases where urllib.request.urlretrieve succeeds but the downloaded file contains invalid JSON. **Perspective 2:** The recall calculation divides by len(evidence_texts) without checking if it's zero, though there's a ternary check. The logic could still result in division by zero in edge cases.
Suggested Fix
Add JSON parsing try-except block after downloading the file.
MEDIUMHardcoded API endpoint URL with potential for key exposure
[redacted]/locomo_bench.py:0
[AGENTS: Vault]secrets
**Perspective 1:** The file contains a hardcoded Anthropic API endpoint URL ('https://api.anthropic.com/v1/messages') in the _llm_call function. While the API key is passed as a parameter, hardcoding the endpoint URL could be problematic if the service changes or if there are security implications with the specific endpoint. Additionally, the function makes direct HTTP requests without proper error handling for credential exposure in logs. **Perspective 2:** The script loads API keys from command-line arguments or environment variables (ANTHROPIC_API_KEY) but doesn't implement secure storage practices. The key is passed around in functions and used in HTTP requests without encryption in transit verification. Additionally, the _load_api_key function exposes the key source in a way that could be logged or leaked. **Perspective 3:** The _llm_call function catches generic exceptions but doesn't specifically handle cases where API keys or sensitive information might be exposed in error messages or stack traces. If the HTTP request fails, error details could potentially leak sensitive information. **Perspective 4:** The code doesn't implement any mechanism for credential rotation or expiration checking. API keys are loaded once and used indefinitely without validation of their age or rotation status.
Suggested Fix
Implement secure credential storage using system keyrings or encrypted configuration files. Ensure API keys are never logged and use HTTPS with certificate validation.
MEDIUMBenchmark processing of conversation data without PII filtering
[redacted]/locomo_bench.py:1
[AGENTS: Warden]privacy
The LoCoMo benchmark script processes conversation data containing detailed personal information (transgender identity, family details, mental health, etc.) without any PII filtering or anonymization. The script ingests and queries this sensitive data, potentially exposing it in retrieval results.
Suggested Fix
Add PII filtering layer before processing benchmark data or use anonymized versions of benchmark datasets.
MEDIUMMissing integrity verification for external model downloads
[redacted]/locomo_bench.py:1
[AGENTS: Supply]supply_chain
**Perspective 1:** The script downloads embedding models (fastembed) without verifying checksums or signatures. This allows MITM attacks or compromised package repositories to inject malicious models. **Perspective 2:** The benchmark script imports chromadb and fastembed without version constraints, allowing incompatible or malicious updates to break reproducibility.
Suggested Fix
Add SHA256 checksum verification for downloaded model files or use signed model repositories.
MEDIUMPotential sensitive data exposure in LLM API calls
[redacted]/locomo_bench.py:1
[AGENTS: Trace]logging
**Perspective 1:** The script makes LLM API calls for room assignments and reranking but doesn't log these operations securely. While API keys are handled, the content sent to LLMs (conversation snippets, questions) could contain sensitive information that should be audited. **Perspective 2:** The benchmark script performs LLM API calls, room assignments, and retrieval operations without structured logging or audit trails. This makes it difficult to trace benchmark execution, debug failures, or verify reproducibility. **Perspective 3:** Each benchmark iteration (conversation + QA pairs) runs without correlation IDs, making it difficult to trace specific failures or performance issues across the distributed operations.
Suggested Fix
Add structured logging with correlation IDs for each benchmark run, including timestamps, API call details (without exposing keys), and retrieval metrics.
MEDIUMInsecure API key handling in benchmark script
[redacted]/locomo_bench.py:1
[AGENTS: Lockdown]configuration
**Perspective 1:** The benchmark script loads API keys from command line arguments and environment variables without proper sanitization or secure storage. API keys are passed to external LLM services and could be exposed in process listings or logs. The script also lacks validation for API key format. **Perspective 2:** The script contains hardcoded model names like 'claude-haiku-4-5-20251001' and 'claude-sonnet-4-6'. If these models are deprecated or have security updates, the script will continue using outdated versions without warning.
Suggested Fix
Use secure credential storage (keyring), mask keys in logs, validate key format, and ensure keys are not passed as command-line arguments where they could appear in shell history.
MEDIUMHallucinated embedding model support without dependency declaration
[redacted]/locomo_bench.py:1
[AGENTS: Provenance]ai_provenance
**Perspective 1:** The code imports and uses `fastembed` for BGE-large embeddings (line 36-53) but there's no dependency declaration for fastembed in the project's pyproject.toml or requirements. The code includes a fallback message 'pip3 install fastembed' but this is not enforced and the benchmark will crash if run with --embed-model bge-large without the dependency installed. **Perspective 2:** The code includes multiple LLM calls to Anthropic API (lines 383-404, 411-445) with hardcoded model names like 'claude-haiku-4-5-20251001' and 'claude-sonnet-4-6'. These model versions are specific and may not be available to all users or may be outdated. The code assumes these endpoints exist and will work without runtime validation.
Suggested Fix
Add fastembed to pyproject.toml optional dependencies or check for import and provide a clear error message with installation instructions.

Summary

Consensus from 144 reviewer(s): Passkey, Vault, Gatekeeper, Deadbolt, Cipher, Specter, Entropy, Egress, Exploit, Harbor, Wallet, Tripwire, Supply, Syringe, Weights, Recon, Lockdown, Phantom, Vector, Tenant, Sanitizer, Trace, Warden, Infiltrator, Siege, Mirage, Prompt, Razor, Gateway, Pedant, Compliance, Fuse, Provenance, Sentinel, Blacklist, Chaos, Gatekeeper, Blacklist, Passkey, Deadbolt, Harbor, Mirage, Exploit, Entropy, Vault, Cipher, Specter, Weights, Supply, Gateway, Lockdown, Phantom, Trace, Egress, Warden, Razor, Wallet, Compliance, Prompt, Syringe, Recon, Sanitizer, Provenance, Tripwire, Siege, Infiltrator, Vector, Chaos, Tenant, Fuse, Pedant, Sentinel, Gatekeeper, Blacklist, Entropy, Passkey, Gateway, Deadbolt, Vault, Exploit, Warden, Cipher, Harbor, Specter, Prompt, Sanitizer, Supply, Weights, Phantom, Siege, Razor, Fuse, Trace, Lockdown, Mirage, Egress, Chaos, Provenance, Tripwire, Syringe, Recon, Pedant, Vector, Infiltrator, Tenant, Compliance, Sentinel, Wallet, Deadbolt, Harbor, Cipher, Vault, Passkey, Entropy, Gateway, Phantom, Supply, Prompt, Blacklist, Gatekeeper, Weights, Tripwire, Specter, Recon, Wallet, Trace, Warden, Egress, Provenance, Infiltrator, Lockdown, Compliance, Pedant, Siege, Syringe, Exploit, Mirage, Sanitizer, Chaos, Vector, Razor, Sentinel, Fuse, Tenant Total findings: 994 Severity breakdown: 37 critical, 135 high, 563 medium, 243 low, 16 info

Note: Fixing issues can create a domino effect — resolving one finding often surfaces new ones that were previously hidden. Multiple scan-and-fix cycles may be needed until you’re satisfied no further issues remain. How deep you go is your call.