Build a Private AI Knowledge Base on Synology DSM 7.4

Synology

Companies are increasingly interested in AI assistants that can answer questions about internal documents, policies, project files, meeting notes, technical manuals, and archived knowledge. The problem is that sending all of that information to a public AI service may not be appropriate.

With DSM 7.4, Synology is moving much closer to a private AI architecture. Synology Drive AI Search can now understand the meaning of files rather than relying only on exact keywords, while Synology AI Runtime can perform embedding, OCR, speech-to-text, and image captioning directly on supported GPU-equipped systems.

That creates an interesting foundation for something more ambitious: a private Retrieval-Augmented Generation, or RAG, knowledge base built around data already stored on Synology.

The key is understanding which pieces Synology provides natively and which pieces you still need to build yourself.

What Is a Private RAG Knowledge Base?

A conventional large language model knows what was included in its training data, but it does not automatically know the contents of your private Synology folders. RAG changes that.

A simplified RAG workflow looks like this:

Your documents

↓

Content extraction

↓

Embeddings

↓

Vector database

↓

Relevant passages retrieved

↓

Local LLM generates the answer

Instead of asking an AI model to answer entirely from memory, the system first retrieves information from your own documents and provides that information to the model as context.

That means you could ask questions such as:

What does our disaster recovery policy say about offsite copies?
Which proposal mentions a 90-day implementation timeline?
What troubleshooting steps did we use for the previous network outage?
Which contract contains this renewal requirement?
What did the project documentation say about the migration order?

The usefulness comes from combining an LLM with information that already belongs to your organization.

DSM 7.4 Already Provides Part of the Retrieval Layer

This is where DSM 7.4 becomes especially interesting. Synology Drive 4.1 AI Search uses semantic indexing rather than relying only on keywords. Synology says its embedding model converts file content into vector representations so Drive can understand context and intent.

That allows a user to search by meaning. You might remember that a document discussed replacing failed storage hardware without remembering whether the actual file contains the words “hardware replacement.”

Semantic search can still surface relevant results.

Synology’s AI stack also includes:

Embedding for understanding document meaning
OCR for extracting text from scans and images
Speech-to-text for making audio and video searchable
Image captioning for describing visual content

Those capabilities run through Synology AI Runtime on supported GPU-equipped systems. That is already remarkably close to the document-processing side of a traditional RAG pipeline.

But Synology Drive AI Search Is Not a Complete RAG System

This distinction matters. Synology Drive AI Search retrieves relevant files using natural language. It does not, by itself, turn Drive into a ChatGPT-style assistant that reads several retrieved passages, reasons across them, and writes a synthesized answer.

A full RAG environment normally adds several components:

Retrieval: Find relevant material.

Context construction: Select the useful passages.

Generation: Send those passages to an LLM.

Answer interface: Present the generated response to the user.

DSM 7.4 can provide increasingly sophisticated retrieval and indexing, but if you want a conversational internal knowledge assistant, you still need the generation layer. That is where local models and Container Manager become useful.

Container Manager Can Host the AI Application Layer

Synology Container Manager allows supported NAS systems to run Docker-based applications and multi-container projects. DSM’s current Project interface supports Docker Compose projects, allowing several containers to be defined and managed together.

A private RAG deployment might therefore include containers for:

A web-based AI interface
A RAG orchestration application
A vector database
Document-processing services
A local model server
Supporting databases

The exact stack can vary.

One user may want a simple document-chat application. Another may build a more sophisticated system that separately manages embeddings, vector storage, access control, and multiple LLMs. The advantage of Container Manager is that those services can live close to the files already stored on the NAS.

Keep the Knowledge Source in Synology Drive

For many deployments, the cleanest architecture is to make Synology Drive Team Folders the authoritative document source.

For example:

Company-Knowledge

Policies
Procedures
Technical Documentation
Projects
Contracts
Training
Product Information

Users can continue managing these documents through ordinary Drive workflows. The AI system then reads from those controlled folders instead of requiring users to upload duplicate files into an unrelated AI application. This reduces one of the biggest problems with internal knowledge bases: multiple copies of the same information becoming inconsistent.

Semantic Indexing Has Important Scope Limits

DSM 7.4 does not semantically index everything on the NAS. Synology currently limits semantic indexing to Synology Drive Team Folders. Hybrid Share folders are excluded. There are also file-size thresholds.

Synology currently specifies semantic-indexing limits of:

30 MB for Office documents and PDFs
200 KB for plain text and Markdown
20 MB for images
100 MB for audio
200 MB for video

When a supported file exceeds its semantic-indexing limit, DSM can fall back to normal file-content indexing rather than semantic indexing. That means folder design matters when preparing a NAS for AI search.

GPU Support Is Currently the Biggest Hardware Limitation

The most exciting DSM 7.4 AI features are not available on every DiskStation.

Synology’s on-device AI models require a supported GPU-equipped storage system. The current AI Runtime beta supports models including the FS3420, FS6420, RS11626xs+, RS3626xs, RS4826xs+, and RS6426xs+.

This is important for owners of mainstream DS Plus systems. You should not assume a DS925+ or DS1825+ will suddenly become a local LLM workstation simply because it runs DSM 7.4. The AI software architecture may be available within DSM, but hardware support determines which on-device models can actually run.

Local AI Also Requires Considerable Memory

AI workloads are much more memory-intensive than ordinary NAS services.

Synology recommends reserving at least 16 GB of system memory when using its GPU-powered AI models.

Its estimated model requirements are:

Embedding: 2 GB
OCR: 2 GB
Speech-to-text: 3 GB
Image captioning: 3 GB

The operating system and other packages need memory in addition to those amounts.

That means AI planning should begin with the complete workload rather than simply asking whether a GPU fits. If the NAS is also handling backup, file services, databases, containers, indexing, and several AI models, memory pressure can become significant.

Container Manager Now Has Limited GPU Access Too

Synology has also begun extending GPU access into Container Manager. Container Manager version 24.0.2-1630 added support for containers to access GPU resources when the NVIDIA Runtime Library is installed. However, that release currently applies only to specific supported models rather than the entire DiskStation portfolio.

This is potentially significant for private AI deployments. It means supported Synology hardware may increasingly be able to run not only Synology’s own AI models but also GPU-aware containerized applications.

However, this should still be treated as a model-specific capability. Do not design a local LLM environment around GPU containers until you have verified that the exact Synology model, GPU, DSM version, Container Manager version, and NVIDIA runtime are supported.

A Practical Synology RAG Architecture

A useful architecture could look like this:

Synology Drive

Stores the authoritative company documents.

↓

DSM 7.4 semantic indexing

Understands documents, scans, images, audio, and video where supported.

↓

Container-based RAG application

Handles chunking, retrieval, prompts, user interface, and conversation history.

↓

Vector database

Stores embeddings and retrieves relevant passages.

↓

Local LLM

Generates an answer using the retrieved content. The user then interacts with a private web interface rather than manually searching individual folders.

You Do Not Have to Run the LLM on the NAS

This is one of the most important architecture decisions. Synology can remain the knowledge and storage layer while another machine provides AI compute.

For example:

Synology NAS

Documents
Permissions
Snapshots
Synology Drive
Backup
Search indexing

Dedicated GPU server

Local LLM
RAG engine
Vector database
AI web interface

The AI server can mount a read-only Synology share over NFS or SMB and build its knowledge index from that data. This is often the more practical architecture for existing Synology Plus owners. You retain DSM’s excellent file-management and protection capabilities without expecting a low-power NAS processor to behave like a dedicated AI workstation.

A Proxmox GPU Server Can Be a Natural Partner

This also fits well with hybrid Proxmox environments.

A Proxmox host can provide a powerful GPU, CPU, and RAM configuration while Synology remains responsible for durable storage.

That architecture might look like:

Users

↓

Local AI interface running on Proxmox

↓

RAG + local LLM

↓

Read-only access to Synology knowledge folders

↓

Synology snapshots and backups

This keeps compute replaceable.

If you later upgrade from one GPU generation to another, your knowledge repository does not need to move. It remains on Synology.

Protect the Vector Database, but Don’t Treat It as the Original Data

A RAG system introduces new data that also deserves protection.

That may include:

Vector databases
Application configurations
Model configuration
Prompts
User settings
Metadata
Conversation history

However, the original Synology files should remain the authoritative source. If the vector index becomes damaged, you should be able to rebuild it from those files. That is much safer than allowing the AI application’s own database to become the only copy of important documents.

Use Read-Only Access Wherever Possible

The first version of an internal RAG assistant generally does not need permission to modify company documents.

Give the AI application read-only access to the folders it needs.

That reduces the consequences of:

A compromised container
Application bugs
Incorrect automation
Prompt injection
Misconfigured permissions

The AI assistant’s job can initially be:

Read → retrieve → answer

rather than:

Read → modify → delete → automate

More powerful agentic capabilities can be evaluated separately after the retrieval environment is stable.

File Permissions Still Matter

A private AI server is only private if access control remains correct.

Imagine a RAG assistant indexes HR, finance, customer, engineering, and general employee documents into one unrestricted vector database.

The underlying Synology permissions may be perfect, but the AI application could effectively bypass them by retrieving information from every indexed folder. For business deployments, the RAG layer therefore needs to respect document access boundaries.

Possible approaches include:

Separate knowledge bases by department
Separate vector collections
User-aware retrieval
Restricted service accounts
Dedicated Team Folders
Read-only shares
Network segmentation

Do not treat authentication as something to add after the AI prototype is already indexing sensitive information.

Keep Private AI Separate From Public API AI

Synology AI Console supports both on-device models and integrations with external AI providers. Supported API integrations include OpenAI, Azure OpenAI, Amazon Bedrock, Google AI Studio, Google Vertex AI, and OpenAI-compatible endpoints. Synology also provides centralized permissions and de-identification options for AI integrations.

Those capabilities are useful, but they are different from a completely local deployment. If your objective is no document content leaving your infrastructure, design the workflow so retrieval and generation both remain local.

If external AI APIs are acceptable, AI Console can provide another architecture. The important point is to decide intentionally rather than assuming every feature labeled “AI” has the same privacy model.

Start With Search Before Building the Full Chatbot

For many organizations, the smartest path is incremental. Start by enabling Synology Drive AI Search where supported. Evaluate whether employees can find information more effectively using natural-language searches.

Then identify the questions users repeatedly ask. Those questions tell you where a RAG assistant might provide additional value. Only then add the vector database, local LLM, and conversational layer. This avoids building an elaborate AI platform before understanding how employees actually search the organization’s information.

DSM 7.4 Makes Synology More Interesting as an AI Data Platform

Synology is not trying to turn every DiskStation into a giant GPU server. The more interesting opportunity is that DSM is becoming an increasingly useful private data layer for AI. Synology Drive already controls documents, permissions, collaboration, versions, and shared folders. DSM 7.4 adds semantic understanding through on-device embeddings, OCR, speech-to-text, and image captioning on supported hardware.

Container Manager provides another layer for deploying custom applications, while newer supported enterprise systems can expose GPU resources to containers. That gives organizations several ways to build private AI. A high-end supported Synology can potentially handle more of the workload directly. A mainstream DiskStation can remain the protected knowledge repository while a dedicated GPU server performs inference. Either way, the valuable data stays under the organization’s control.

The best architecture is therefore not necessarily “run everything on Synology.”

It is:

Keep the knowledge on Synology, place AI compute where it performs best, restrict access carefully, and maintain independent backups of the original data. A Synology security check-up can identify access weaknesses before deployment.

About Epis Technology

Epis Technology helps businesses design Synology infrastructure that can support modern private AI and knowledge-management workflows without compromising storage reliability or data protection. Services include DSM 7.4 deployment, Synology Drive, AI Search planning, GPU-capable Synology architecture, Container Manager, virtualization and Proxmox integration, SMB and NFS storage, backup design, snapshots, Microsoft 365 and Google Workspace protection, networking, ransomware resilience, and disaster recovery. Epis Technology can help determine whether AI workloads should run directly on supported Synology hardware or use Synology as the secure data layer for a dedicated local AI compute platform.