Background daemons

Background daemons are standalone Node.js services that run alongside the Comindwork platform. They handle recurring tasks like file synchronization, email polling, and messaging bot integrations.

Architecture

Background daemons are not part of the app engine - they run as separate processes managed by PM2 or system cron.

Available integrations:

Daemon	Purpose
sftp-sync	Synchronize files between Comindwork and external SFTP servers
gmail-sync	Poll Gmail accounts and create records from incoming emails
telegram-bot	Relay messages between Telegram chats and Comindwork records
data-import	Batch import data from external sources on a schedule
demo-data	Generate sample data for demo workspaces

Building a background daemon

A background daemon is a Node.js script that:

1. Connects to the Comindwork API using service credentials
2. Performs its task (poll, sync, transform)
3. Exits or sleeps until the next cycle

Project structure

ext-integrations/
└── my-daemon/
    ├── package.json
    ├── index.js            # Entry point
    ├── config.js            # Environment-specific settings
    └── ecosystem.config.js  # PM2 configuration

PM2 configuration

Use PM2 to manage daemon processes in production:

// ecosystem.config.js
module.exports = {
  apps: [{
    name: "my-daemon",
    script: "./index.js",
    cron_restart: "*/15 * * * *",  // restart every 15 minutes
    autorestart: false,
    env: {
      NODE_ENV: "production",
      CMW_API_URL: "https://your-instance.comind.work",
      CMW_API_TOKEN: "your-service-token"
    }
  }]
};

Connecting to the API

Daemons use the Comindwork REST API to read and write data:

const axios = require("axios");

const api = axios.create({
  baseURL: process.env.CMW_API_URL + "/api",
  headers: {
    Authorization: `Bearer ${process.env.CMW_API_TOKEN}`
  }
});

// Example: fetch records
const response = await api.get("/records/WORKSPACE/APP", {
  params: { filter: 'state="open"', limit: 50 }
});

SFTP sync example

The SFTP sync daemon monitors a remote SFTP server and creates or updates file records in Comindwork when new files appear.

Typical workflow:

1. Connect to the SFTP server using configured credentials
2. List files in the monitored directory
3. Compare against existing Comindwork records (by filename or hash)
4. Download new files and create records with attachments
5. Optionally move processed files to an archive directory

Gmail sync

The Gmail sync daemon polls a Gmail account via the Gmail API and creates records from incoming messages.

Workflow:

1. Authenticate with Gmail using OAuth2 or service account credentials
2. Query for unread messages matching configured filters (labels, senders, subjects)
3. Parse email content (subject, body, attachments)
4. Create a new record in the target app with email data mapped to fields
5. Mark the email as processed (label or archive)

Deployment

Using PM2

# Start the daemon
pm2 start ecosystem.config.js

# Monitor running daemons
pm2 list
pm2 logs my-daemon

# Set up PM2 to survive reboots
pm2 startup
pm2 save

Using system cron

For simple one-shot tasks, use cron instead of PM2:

# Run every 15 minutes
*/15 * * * * cd /path/to/my-daemon && node index.js >> /var/log/my-daemon.log 2>&1

Best practices

• Idempotency - design each run to be safe to repeat. Use unique keys (filenames, email IDs) to detect duplicates.
• Error handling - log errors and continue processing remaining items. A single bad record should not stop the entire sync.
• Rate limiting - respect Comindwork API rate limits. Add delays between batch operations.
• Configuration - use environment variables for all connection details. Never hardcode credentials.
• Monitoring - use PM2 monitoring or external health checks to detect daemon failures.