Building a Secure RAG Pipeline in n8n with SharePoint and Azure (Data Stays Inside)

Who This Is For

You have a large pile of internal documents. Policy manuals, legal contracts, compliance frameworks, SOPs. Probably somewhere between 5,000 and 10,000 pages of material your team should be able to search and get answers from.

You've looked at ChatGPT. You've looked at Microsoft Copilot. The immediate problem: your legal or IT team has already told you that routing that content through a third-party cloud is off the table.

This post walks through exactly how to build a retrieval-augmented generation (RAG) system over those documents using tools that keep everything inside your existing Azure perimeter. n8n for orchestration, Microsoft Graph API for SharePoint access, Azure OpenAI for embeddings, and either a self-hosted Qdrant instance or Azure AI Search for the vector store.

Nothing leaves Azure. Nothing touches OpenAI's public endpoints.

The Architecture Before Any Code

Before getting into the n8n specifics, here's the full data flow:

1. Documents sit in SharePoint. n8n pulls them using the Microsoft Graph API with an Azure AD app registration.
2. Each document gets chunked into overlapping text segments.
3. Each chunk gets converted to a vector embedding using Azure OpenAI's text-embedding-ada-002 deployment (running inside your Azure subscription, not OpenAI's public cloud).
4. Those vectors get stored in Qdrant (self-hosted on a VM or container inside your VNet) or Azure AI Search.
5. When a user asks a question, n8n embeds the query using the same Azure OpenAI deployment, queries the vector store for the closest matching chunks, then passes those chunks to Azure OpenAI's GPT-4 deployment to generate an answer.

Two important things that get missed: the embedding step and the query step both use Azure OpenAI, not the public api.openai.com endpoint. Azure OpenAI is a separate deployment inside your subscription. Data doesn't cross to OpenAI's infrastructure.

Setting Up the Azure AD App Registration

This is where most people hit their first wall.

To call the Microsoft Graph API from n8n, you need an Azure AD app registration with the right permissions. Go to the Azure portal, Entra ID (formerly Azure Active Directory), App registrations, New registration.

The permissions you need for SharePoint document access:

• Sites.Read.All (application permission, not delegated) to list sites and drive items
• Files.Read.All (application permission) to download file content

Use application permissions, not delegated. Delegated permissions require a user to be logged in interactively. Application permissions let n8n call the Graph API as the app itself, with no user session.

After adding the permissions, click "Grant admin consent." Without that step, the permissions exist but aren't active.

Then go to Certificates and Secrets, add a client secret. Copy it immediately. Azure won't show it again.

Note down: the Tenant ID, Client ID, and Client Secret. You'll need all three.

Handling Auth Token Refresh in n8n

The Microsoft Graph API uses OAuth 2.0 with short-lived access tokens. Tokens expire after about 3,600 seconds. If your ingestion workflow runs longer than an hour (which it will for a full 10,000-page corpus), the token expires mid-run.

The right way to handle this in n8n: don't cache the token manually. Use n8n's built-in HTTP Request node with the OAuth2 credential type set to "Client Credentials."

Set it up this way:

• Authorization URL: https://login.microsoftonline.com/{your-tenant-id}/oauth2/v2.0/authorize
• Access Token URL: https://login.microsoftonline.com/{your-tenant-id}/oauth2/v2.0/token
• Scope: https://graph.microsoft.com/.default
• Grant type: Client Credentials
• Client ID and Secret from your app registration

When you configure the HTTP Request node to use this credential, n8n handles token refresh automatically. When the token expires, it fetches a new one before the next request. You don't need to write any refresh logic yourself.

Pulling Documents From SharePoint

The Microsoft Graph API endpoint to list all files in a SharePoint document library:

GET https://graph.microsoft.com/v1.0/sites/{site-id}/drives/{drive-id}/root/children

To get the site-id, call:

GET https://graph.microsoft.com/v1.0/sites?search=*

That returns a list of all accessible SharePoint sites. Pick the one you want, grab its ID.

Then list drives on that site:

GET https://graph.microsoft.com/v1.0/sites/{site-id}/drives

Pick the "Documents" drive, get its ID. Then you can walk the folder tree with the children endpoint.

For each file, the item object includes a @microsoft.graph.downloadUrl property. That's a pre-authenticated download URL. Send a GET request to that URL and you get the raw file content. Works for PDFs, Word docs, and plain text files.

In n8n, this is two HTTP Request nodes: one to list items, one to download content. Add a SplitInBatches node between them if you have hundreds of documents so you don't hit Graph API rate limits (the default limit is 10,000 requests per 10 minutes per app, which is generous, but large file downloads can slow things down).

Chunking Strategy for Legal and Policy Documents

Chunking is the part that most tutorials treat as an afterthought. The chunk size you choose changes retrieval quality dramatically.

For long legal or policy documents, a 1,024-token chunk with a 128-token overlap works better than the commonly suggested 512-token chunks. Policy documents have long definitional clauses. Cutting them at 512 tokens means you're regularly cutting a clause in the middle. The retrieved chunk doesn't make sense on its own and the LLM can't answer well from it.

1,024 tokens gets you roughly 750-800 words per chunk. That's enough for a complete clause or paragraph with context on either side. The 128-token overlap means the boundary between chunks always has some shared context so nothing falls through the gap between two adjacent chunks.

In n8n, you'll implement chunking in a Code node. The logic: take the full document text, split by tokens or by character count (characters are easier, use 4,000 characters per chunk with 512 overlap as an approximation of the 1,024/128 token split).

One specific thing: strip page headers and footers before chunking. They add noise. A legal document with "Page 47 of 312 | CONFIDENTIAL | Company Name | Document Revision 3.1" appearing in every chunk means those strings end up in your embeddings and interfere with semantic similarity.

Generating Embeddings With Azure OpenAI

Once you have a chunk, you call the embeddings endpoint. The key difference from using public OpenAI: the base URL.

Public OpenAI endpoint: https://api.openai.com/v1/embeddings Azure OpenAI endpoint: https://{your-resource-name}.openai.azure.com/openai/deployments/{deployment-name}/embeddings?api-version=2024-02-01

In n8n, use an HTTP Request node with this URL structure. Set the api-key header to your Azure OpenAI key (not a Bearer token, Azure OpenAI uses api-key as the header name). Pass the chunk text as the input field in the request body.

You'll need to have deployed the text-embedding-ada-002 model in your Azure OpenAI resource beforehand. The deployment name is whatever you called it when you created it in Azure AI Foundry (previously called Azure OpenAI Studio).

The response includes an embedding array of 1,536 floats. That's your vector for the chunk.

For a 10,000-page corpus broken into 1,024-token chunks, you're looking at roughly 40,000 to 60,000 chunks depending on document density. At $0.0001 per 1,000 tokens, generating all embeddings from scratch costs about $4 to $6. Run this once, store the vectors. You don't re-embed every time someone asks a question.

Storing Vectors in Qdrant

Qdrant is a vector database. You can run it as a Docker container on any VM inside your Azure VNet.

docker run -p 6333:6333 qdrant/qdrant:v1.9.2

Version 1.9.2 specifically has better memory usage for large collections compared to earlier 1.x versions.

Create a collection with 1,536 dimensions (matching the output size of text-embedding-ada-002):

PUT /collections/enterprise-docs
{
  "vectors": {
    "size": 1536,
    "distance": "Cosine"
  }
}

In n8n, store each chunk as a Qdrant point. Each point has:

• An ID (you can use a hash of the document ID and chunk index)
• The 1,536-float vector
• A payload with the original chunk text, the source document filename, the page number, and the site/library it came from

Always store the source metadata in the payload. When the system returns an answer, users will want to know "where did this come from" and you'll need to surface the SharePoint link or document name.

The Query Flow in n8n

The ingestion side runs once (and then incrementally as documents update). The query side runs every time a user asks a question.

The flow:

1. Trigger (webhook, Teams bot, chat interface, whatever fits your setup)
2. HTTP Request node calls Azure OpenAI embeddings with the user's question text
3. HTTP Request node sends the query vector to Qdrant's search endpoint: POST /collections/enterprise-docs/points/search with the query vector and limit: 5
4. The 5 closest chunks come back with their payloads (the original text)
5. Build a prompt: system message explaining the task, then the 5 chunks as context, then the user's question
6. HTTP Request node calls Azure OpenAI chat completions: https://{resource}.openai.azure.com/openai/deployments/{gpt4-deployment}/chat/completions?api-version=2024-02-01
7. Return the answer, plus the source document names from the chunk payloads

The whole query chain runs in under 4 seconds for most questions. Embedding the query takes about 200ms. Qdrant search over 50,000 vectors takes under 50ms. The GPT-4 completion is the slow part, usually 2-3 seconds.

Keeping the Index Fresh

Documents get updated. New SOPs get added. Old policies get replaced.

The simplest incremental update approach: store a last_modified timestamp alongside each document when you ingest it. Run a scheduled n8n workflow every night that calls the Graph API with a delta query:

GET https://graph.microsoft.com/v1.0/sites/{site-id}/drives/{drive-id}/root/delta?$select=id,name,lastModifiedDateTime,file

The delta endpoint returns only items that changed since your last token. Compare lastModifiedDateTime to your stored timestamps. Re-chunk and re-embed only the changed documents. Delete the old Qdrant points for those documents (by filtering on the document ID in the payload), then insert the new ones.

This keeps the index fresh without re-processing your entire corpus every night.

The Permissions Question

People worry about document-level access control. If you have 50 departments and each SharePoint library has different read permissions, you probably don't want every user getting chunks from documents they're not supposed to see.

There are two ways to handle this:

Option A: build separate Qdrant collections per department or access tier. The query flow checks the user's role first, then queries only the collections they have access to.

Option B: store access metadata (allowed groups or departments) in the Qdrant payload, then filter at query time using Qdrant's payload filters. This keeps everything in one collection but restricts which chunks come back for a given user.

Option B is cleaner at scale. Qdrant's filter syntax supports array membership, so "must": [{"key": "allowed_groups", "match": {"any": ["legal", "compliance"]}}] returns only chunks tagged for those groups.

What This Actually Takes to Build

If you've done API integrations before, the full pipeline from scratch takes about two weeks. Roughly: two days on the Azure AD setup and Graph API pagination, three days on the chunking and embedding pipeline, three days on Qdrant setup and the query flow, the rest on testing edge cases (PDFs with embedded images, documents with unusual encoding, handling very long documents that produce hundreds of chunks).

If this is new territory, it takes longer and there are more things that break in unexpected ways. The Graph API returns paginated results using @odata.nextLink tokens. Qdrant's batch upload has a payload size limit. Azure OpenAI rate limits are per-deployment-per-minute and you'll hit them during initial ingestion if you don't add a throttle.

Doing it once teaches you a lot. Doing it a second time is much faster.

---

If you want us to build this for you, book a free workflow audit at airtective.com