Troubleshooting

Solutions for common issues across all SmartPay services.

Quick Diagnostics

Check Service Health

curl http://localhost:8080/health  # Gateway
curl http://localhost:8000/health  # Agent

View Logs

docker compose logs -f gateway wallet-core agent

Check Database

psql $DATABASE_URL -c "SELECT 1"

Gateway Issues

Connection to Wallet-Core Failed

Symptoms

Gateway health shows wallet_core: unavailable
500 errors on payment endpoints
Logs show gRPC connection errors

Causes & Solutions:

Cause	Solution
Wallet-Core not running	`docker compose up wallet-core`
Wrong address	Check `WALLET_CORE_ADDRESS` in config
TLS mismatch	Verify TLS settings match on both services
Network issue	Check firewall rules, verify network connectivity

# Verify Wallet-Core is running
docker compose ps wallet-core

# Check Gateway config
grep WALLET_CORE gateway/.env

401 Unauthorized

Symptoms

All API requests return 401
"Invalid token" or "Unauthorized" in response

Causes & Solutions:

Cause	Solution
Missing header	Include `Authorization: Bearer <token>`
Invalid API key	Verify key exists and is active
Expired JWT	Refresh token or re-authenticate
Wrong secret	Check `JWT_SECRET` matches between services

# Test with API key
curl -H "Authorization: Bearer olive_live_xxx" \
  http://localhost:8080/api/v1/health

# Check if key exists in database
psql $DATABASE_URL -c "SELECT * FROM api_keys WHERE key_hash = ..."

403 Forbidden

Symptoms

Valid auth but access denied
"Insufficient permissions" error

Causes & Solutions:

Cause	Solution
Wrong role	User lacks required role for endpoint
Wrong scopes	API key missing required scope
PEP restriction	PEP access requires OTP verification

429 Rate Limited

Symptoms

Requests start failing after high volume
X-RateLimit-Remaining: 0 in response

Solution:

# Increase rate limits in gateway/config.yaml
rate_limit:
  requests_per_second: 200  # Increase from 100
  burst: 400                # Increase from 200

Agent-TS Issues

OpenAI API Errors

Symptoms

Agent not responding to messages
"OpenAI API error" in logs
Timeout errors

Causes & Solutions:

Cause	Solution
Invalid API key	Verify `OPENAI_API_KEY` in `.env`
Quota exceeded	Check OpenAI billing/usage
Network blocked	Verify outbound HTTPS to api.openai.com
Rate limited	Add retry logic or reduce request rate

# Test OpenAI connectivity
curl https://api.openai.com/v1/models \
  -H "Authorization: Bearer $OPENAI_API_KEY"

KYC Image Processing Failed

Symptoms

"Invalid image" or "Processing failed" messages
OCR returns empty results
S3 upload errors

Causes & Solutions:

Cause	Solution
Missing data URI	Ensure `media.data` includes `data:image/jpeg;base64,...`
Invalid format	Only JPEG and PNG supported
AWS credentials	Verify `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
S3 permissions	Check bucket policy allows PutObject

# Test S3 access
aws s3 ls s3://${AWS_S3_BUCKET_NAME}

# Verify image format
echo "$IMAGE_DATA" | head -c 50
# Should start with: data:image/jpeg;base64,

Assistant Not Calling Functions

Symptoms

Agent responds but doesn't perform actions
No function calls in logs
Generic responses instead of wallet operations

Causes & Solutions:

Cause	Solution
Assistant not created	Check `OPENAI_ASSISTANT_ID` is set
Tools not registered	Verify tools in `smartpay-agent.ts`
Context lost	Check conversation history management

# Check assistant exists
curl https://api.openai.com/v1/assistants/${OPENAI_ASSISTANT_ID} \
  -H "Authorization: Bearer $OPENAI_API_KEY"

Wallet-Core Issues

Database Connection Failed

Symptoms

Service won't start
"Failed to connect to database" in logs
Migrations fail

Causes & Solutions:

Cause	Solution
Invalid DSN	Check `DATABASE_URL` format
Database not running	Start PostgreSQL container
Network issue	Verify database host is accessible
Wrong credentials	Confirm username/password

# Test database connection
psql $DATABASE_URL -c "SELECT version()"

# Check DATABASE_URL format
# postgresql://user:password@host:5432/database

Migrations Failed

Symptoms

Tables don't exist
Schema mismatch errors
"relation does not exist" SQL errors

Causes & Solutions:

Cause	Solution
Permissions	Ensure user has CREATE TABLE permission
Existing tables	Check for conflicting tables
Migration error	Check logs for specific error

# Manually run migrations
cd wallet-core
go run cmd/migrate/main.go -config config.yaml

# Check table existence
psql $DATABASE_URL -c "\dt"

Duplicate Transactions

Symptoms

Same transaction appears multiple times
Balance incorrect

Causes & Solutions:

Cause	Solution
Reused request_id	Generate unique UUID for each request
Client retry	Implement proper retry with same request_id
Missing idempotency	Always include `request_id` in payments

Info

SmartPay uses idempotency keys. If you retry with the same request_id, the original transaction is returned instead of creating a duplicate.

Common Error Codes

Code	Name	Meaning	Action
400	Bad Request	Invalid request body	Check request format and required fields
401	Unauthorized	Invalid/missing auth	Verify credentials
402	Payment Required	Insufficient balance	Check user balance before payment
403	Forbidden	Access denied	Check user role and permissions
404	Not Found	Resource doesn't exist	Verify ID/phone number
409	Conflict	Duplicate resource	Resource already exists
422	Unprocessable	Validation failed	Check field constraints
429	Too Many Requests	Rate limited	Reduce request rate
500	Internal Error	Server error	Check server logs
503	Unavailable	Service down	Check dependent services

Log Analysis

Gateway Logs

# Filter by request ID
docker compose logs gateway | grep "request_id=abc123"

# Filter errors only
docker compose logs gateway | grep "level=error"

# Filter by endpoint
docker compose logs gateway | grep "POST /api/v1/payments"

Common Log Patterns

Pattern	Meaning
`level=error`	Error occurred
`status=401`	Authentication failed
`status=500`	Internal error
`grpc_error`	Wallet-Core communication issue
`timeout`	Request took too long

Performance Issues

Slow Response Times

Symptoms

API responses take > 1 second
Timeouts on some requests

Diagnosis & Solutions:

# Check database query times
psql $DATABASE_URL -c "SELECT * FROM pg_stat_statements ORDER BY mean_time DESC LIMIT 5"

# Check connection pool
psql $DATABASE_URL -c "SELECT count(*) FROM pg_stat_activity"

# Add missing indexes
psql $DATABASE_URL -c "CREATE INDEX idx_transactions_user_id ON transactions(user_id)"

Getting Help

Check Logs

Always check service logs first

Health Endpoints

Use /health for quick diagnostics

GitHub Issues

Search existing issues for solutions

Documentation

Review relevant docs sections

Quick Diagnostics​

Gateway Issues​

Connection to Wallet-Core Failed​

401 Unauthorized​

403 Forbidden​

429 Rate Limited​

Agent-TS Issues​

OpenAI API Errors​

KYC Image Processing Failed​

Assistant Not Calling Functions​

Wallet-Core Issues​

Database Connection Failed​

Migrations Failed​

Duplicate Transactions​

Common Error Codes​

Log Analysis​

Gateway Logs​

Common Log Patterns​

Performance Issues​

Slow Response Times​

Getting Help​

Quick Diagnostics

Gateway Issues

Connection to Wallet-Core Failed

401 Unauthorized

403 Forbidden

429 Rate Limited

Agent-TS Issues

OpenAI API Errors

KYC Image Processing Failed

Assistant Not Calling Functions

Wallet-Core Issues

Database Connection Failed

Migrations Failed

Duplicate Transactions

Common Error Codes

Log Analysis

Gateway Logs

Common Log Patterns

Performance Issues

Slow Response Times

Getting Help