Epithium
/
vidrip
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
vidrip/CLAUDE.md

8.5 KiB
Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Development Commands

# Install dependencies
npm run install:all

# Development (run in separate terminals)
npm run dev:backend    # Starts on :3001 with tsx watch
npm run dev:frontend   # Starts on :3000 with Vite HMR

# Within backend/ directory
cd backend
npm run dev      # Development with tsx watch
npm run build    # TypeScript compilation
npm start        # Run compiled version

# Within frontend/ directory
cd frontend
npm run dev      # Vite dev server
npm run build    # TypeScript + Vite build
npm run preview  # Preview production build

Architecture Overview

VidRip is a monorepo with separate backend (Express/SQLite) and frontend (React) apps.

Core Scheduler Logic

The scheduler in backend/src/services/scheduler.ts operates on two separate timers:

  1. Channel Checker: Cron job (0 * * * *) runs every hour to check active channels for new videos
  2. Download Cycle: Self-scheduling recursive setTimeout that:
    • Downloads one pending video
    • Calculates next run: intervalHours * 60 ± random(varianceMinutes)
    • Schedules itself again with the calculated delay

The variance prevents predictable download patterns. The scheduler uses a single global isDownloading flag to prevent concurrent downloads.

Database Layer

SQLite database (backend/data.db) with three tables managed via better-sqlite3:

  • channels: YouTube channels to monitor
  • videos: Individual videos with status tracking (pending/downloading/completed/failed)
  • config: Key-value store for app settings

All database operations are in backend/src/db/database.ts using synchronous better-sqlite3 API. The database auto-initializes on first run.

yt-dlp Integration

backend/src/services/ytdlp.ts wraps the yt-dlp CLI (must be installed on system):

  • Uses child_process.spawn to call yt-dlp
  • --dump-json extracts metadata without downloading
  • --flat-playlist lists channel videos efficiently
  • Progress tracking via stdout parsing (looks for percentage patterns)
  • Downloads merge best video+audio to MP4 format

Frontend Proxy Setup

Vite development server (port 3000) proxies to backend (port 3001):

  • /api/* → Backend API endpoints
  • /downloads/* → Static video files

This is configured in frontend/vite.config.ts. Production deployments need a reverse proxy (nginx, etc.).

API Structure

Routes are split by domain:

  • backend/src/routes/channels.ts: Channel CRUD + refresh
  • backend/src/routes/videos.ts: Video CRUD + retry + streaming
  • backend/src/routes/config.ts: Settings + scheduler control + WebDAV test

Video files are served via /api/videos/:id/stream endpoint which handles both local storage and WebDAV proxying.

Key Constraints

  • Only 1 concurrent download supported (hardcoded in scheduler)
  • Download progress is ephemeral (not persisted to DB, only in-memory)
  • No authentication/authorization
  • yt-dlp must be in PATH (external dependency)
  • Database uses WAL mode by default for better concurrency

Important Implementation Notes

Scheduler Restart Behavior

When config changes (e.g., interval or variance), call restartScheduler() from backend/src/services/scheduler.ts. This stops the cron task and clears the setTimeout chain, then restarts with new settings.

Video Status State Machine

Videos progress through states: pending → downloading → completed/failed

  • Retry functionality resets failed back to pending
  • Deleting a channel cascades to delete all its videos and files

Channel Refresh

Adding a channel fetches ALL videos from that channel using yt-dlp's --flat-playlist. Large channels (1000+ videos) may take time. Videos are created in pending state and downloaded according to the schedule.

File Locations

  • Database: backend/data.db
  • Downloaded videos (local mode): backend/downloads/{videoId}.mp4
  • Downloaded videos (WebDAV mode): Stored on WebDAV server, proxied through backend
  • Environment: Optional backend/.env (PORT, NODE_ENV)

System Requirements

  • Node.js 18+
  • yt-dlp installed and in PATH (pip install yt-dlp or brew install yt-dlp)

Troubleshooting 403 Errors

If you encounter 403 Forbidden errors when downloading videos:

Built-in Mitigations

The app automatically includes these anti-403 measures:

  • Browser-like User-Agent headers
  • YouTube referer
  • Multiple retry attempts (5 extractor retries, 10 connection retries)
  • Request throttling (1-5 second delays between requests)
  • Fragment retries for long videos

Additional Solution: Browser Cookies

For persistent 403 errors, export cookies from your browser:

  1. Install browser extension:

    • Chrome/Edge: "Get cookies.txt LOCALLY"
    • Firefox: "cookies.txt"

  2. Export YouTube cookies:

    • Visit youtube.com while logged in
    • Click extension icon and export cookies
    • Save as cookies.txt

  3. Add to VidRip:

    • Place cookies.txt in backend/ directory
    • VidRip will automatically use it if present
    • No restart needed

  4. Keep cookies fresh:

    • YouTube cookies expire periodically
    • Re-export if 403 errors return
    • Consider using an account with YouTube Premium to avoid restrictions

Other Tips

  • Update yt-dlp: pip install --upgrade yt-dlp (YouTube changes frequently)
  • Increase download interval to avoid rate limiting
  • Use variance to randomize download times
  • Avoid downloading too many videos from the same channel quickly

WebDAV Storage

VidRip supports storing videos on a WebDAV server instead of local storage. This is useful for:

  • Network-attached storage (NAS) devices
  • Cloud storage with WebDAV support (Nextcloud, ownCloud)
  • Remote servers with WebDAV access

Enabling WebDAV

  1. Go to Settings page in the UI
  2. Enable "WebDAV Storage"
  3. Configure connection details:
    • Server URL: Full WebDAV server URL (e.g., https://nextcloud.example.com/remote.php/dav/files/username)
    • Username: WebDAV authentication username
    • Password: WebDAV authentication password
    • Directory Path: Subdirectory on server (default: /vidrip/)
  4. Click "Test Connection" to verify settings
  5. Save Settings

How WebDAV Works

Download Flow:

  1. Video is downloaded to backend/downloads/ temporarily
  2. After successful download, video is uploaded to WebDAV server
  3. Local temporary file is deleted after successful upload
  4. Database stores path as webdav://{videoId}.mp4

Playback Flow:

  1. Frontend requests /api/videos/:id/stream
  2. Backend detects WebDAV storage (path starts with webdav://)
  3. Backend creates stream from WebDAV server
  4. Stream is proxied to frontend (transparent to user)

Deletion Flow:

  • When deleting a video, backend automatically removes from WebDAV server
  • Database record is also deleted

Supported WebDAV Servers

Tested with:

  • Nextcloud: Use WebDAV URL format: https://your-nextcloud.com/remote.php/dav/files/username
  • ownCloud: Similar to Nextcloud
  • Generic WebDAV: Any RFC-compliant WebDAV server

Configuration Details

WebDAV settings are stored in the config table:

  • webdavEnabled: 'true' or 'false'
  • webdavUrl: Server URL
  • webdavUsername: Username
  • webdavPassword: Password (stored in plaintext in database)
  • webdavPath: Directory path on server

Troubleshooting WebDAV

Connection Test Fails:

  • Verify server URL is correct and accessible from backend server
  • Check username/password are correct
  • Ensure WebDAV path exists or backend has permission to create it
  • Check firewall rules if server is remote

Upload Fails:

  • Check available disk space on WebDAV server
  • Verify write permissions on directory
  • Check WebDAV server logs for detailed errors
  • Ensure network is stable for large uploads

Playback Issues:

  • Verify WebDAV server is accessible from backend
  • Check network bandwidth between backend and WebDAV
  • Browser console may show streaming errors
  • Try downloading video directly from WebDAV to verify file integrity

Migration:

  • Existing local videos remain in backend/downloads/
  • New videos go to WebDAV when enabled
  • To migrate existing videos, manually upload to WebDAV and update database filePath
  • Or delete and re-download videos

Security Considerations

  • Credentials stored in local database (plaintext)
  • Backend must have network access to WebDAV server
  • Use HTTPS for WebDAV URL to encrypt credentials in transit
  • Consider firewall rules to restrict WebDAV access to backend IP only
  • WebDAV server should require authentication