Azure Data Factory consulting services
Senior US-based engineers build the pipelines that sit underneath your reporting: every source connected once, data landed in a staging area on a schedule you choose, shaped into the tables your Power BI semantic model expects, and alerted when a run fails instead of failing quietly at 3am. Built in your own Azure subscription, published from source control, one written price, typical delivery in 2–4 weeks with 30 days of hypercare.
Representative dashboard — sample data
What you get
A dashboard is only as trustworthy as the load that ran the night before. This is the part of the stack nobody demos, and the part that decides whether anyone believes the numbers.
Each system becomes a linked service and a set of datasets, defined in one place. New reporting reuses the connection instead of adding another private query with its own credentials and its own opinion of the data.
Copying the full history every night works until it doesn't. Pipelines track a watermark column and move only what changed, so the load window stops growing every year your business does.
Some data is nightly, some has to run when a file lands, some has to catch up cleanly after an outage. Time schedules, tumbling windows, and storage-event triggers each get used where they actually fit.
A self-hosted integration runtime runs inside your network and dials out to Azure. Your SQL Server, ERP database, or file share stays exactly where it is, and the firewall change request stays unwritten.
Every pipeline gets retry rules, a run history you can read, and an alert to a real inbox or Teams channel when something breaks — so a bad load is caught before a leadership meeting finds it first.
The factory is wired to a Git repository, so changes are reviewable and a development branch can be tested before it reaches production. You keep the repository, the code, and the deployment templates.
How it works
A free consultation covers what systems hold the data, how each one can be read, how fresh the numbers need to be, and what breaks today. You get a written scope, one fixed price, and a delivery date.
Linked services and integration runtimes go in, copy pipelines land raw data in staging, and transformation steps turn it into the model-ready tables your reports read. Each batch is reconciled against the source before sign-off.
Triggers, dependencies, retries, and alerting are configured, everything is published from source control, and you get run-book documentation, a walkthrough, and 30 days of hypercare.
Azure Data Factory is Microsoft's managed data integration service: you define linked services that point at your systems, datasets that describe what to read and write, and pipelines made of activities that copy and transform the data on a schedule. Microsoft's own Azure Data Factory documentation (opens in new tab) describes it as a serverless service for building extract-transform-load and extract-load-transform workflows — meaning there is no server for you to size, patch, or keep running, and it runs entirely in your own subscription.
Our engagement is the build, not a licence resale or an advisory deck. Deliverables are the pipelines themselves, the staging area they load into, the triggers that run them, the alerting that watches them, the source-control repository they publish from, and written documentation of what runs when and what to do when a run fails. The work is performed remotely by senior US-based engineers from our Orange County, CA base, serving every US state, at a fixed fee agreed after your free consultation.
Most reporting problems that look like Power BI problems are load problems. A measure disagrees with the finance system because the extract ran before the close entries posted; a dashboard is blank on Monday because Friday's file never arrived and nobody was told. Data Factory is where those problems are fixed, because it owns the order of operations: land the source, validate it, shape it, then refresh the model — never the other way round.
That makes this the layer beneath the reporting work rather than a replacement for it. If the reports themselves still need building, our fixed-fee Power BI dashboard development service covers semantic modeling, DAX, and the governed workspace they live in. If your current reporting still runs on an on-premises report server, our SSRS to Power BI migration service moves the reports and subscriptions, and pipelines are frequently the thing that has to exist first so the new model has something clean to read.
The first practical question on nearly every project is how Azure will reach data that lives in your building. The answer is the integration runtime — the compute Data Factory uses to run an activity. Microsoft's integration runtime documentation (opens in new tab) sets out the three kinds: the managed Azure runtime for cloud sources, the self-hosted runtime you install on a machine in your own network for private data, and the Azure-SSIS runtime for executing existing SQL Server Integration Services packages.
The self-hosted runtime is the one that decides most on-premises projects. It connects outbound to Azure, so a database behind your firewall can be read without publishing it to the internet or opening an inbound port. We install and register it, agree with your IT team which machine it runs on, and document the patching and high-availability options, because an unmonitored runtime on a forgotten server is the single most common way a working pipeline estate quietly stops working a year later.
Every estate is different, but sources fall into a small number of shapes. The table below is the pattern library we work from; the inventory in your written scope is a filled-in version of it.
| Where the data lives | How Data Factory reaches it | What gets built |
|---|---|---|
| On-premises SQL Server, ERP database, or file share | Self-hosted integration runtime installed inside your network, connecting outbound | Watermark-based incremental copy into a staging area, plus a schedule and a failure alert per pipeline |
| Azure SQL Database, Azure Storage, or another cloud database | Managed Azure integration runtime — nothing to install | Parameterized copy activities driven by a control table, so adding a table is configuration rather than a new pipeline |
| SaaS applications with a REST or OData API | Built-in connector where one exists, otherwise a REST dataset with paging | Credential storage in Key Vault, paged extraction, retry handling, and de-duplication on re-run |
| Files delivered by a partner (SFTP, CSV, Excel) | Storage event trigger on arrival, or a scheduled sweep of the drop folder | Schema validation, a quarantine folder for rejected files, and an alert naming the file that failed |
| Existing SSIS packages you would rather not rewrite | Azure-SSIS integration runtime executing the packages as they are | A per-package recommendation: lift what earns its keep, rebuild the thin wrappers as native pipelines |
| The Power BI semantic model at the end of the chain | A refresh step sequenced after the load completes successfully | Dependency-aware ordering, so a report is never refreshed on top of a load that failed halfway |
A pipeline that runs is easy. A pipeline estate somebody can still operate in eighteen months takes three extra pieces of work, and they are the ones a rushed build leaves out. The first is the trigger design: Microsoft's guide to pipelines and activities (opens in new tab) describes how activities are chained and how dependencies decide what runs after a success, a failure, or a skip. Getting those conditions right is the difference between a failed extract stopping the chain and a failed extract silently publishing yesterday's numbers as today's.
The second is monitoring. Run history, retry policies, and alert rules are configured so a failure reaches a person, with enough detail to say which activity broke and on which source. The third is source control and promotion: the factory is wired to a Git repository, changes are made on a branch, and publishing to production is a deliberate act with a deployment template rather than someone editing live pipelines in a browser. If the distribution of results is its own project — approvals, notifications, and routing downstream of the load — that belongs with our Power Automate consulting service rather than in the factory. Event-driven, transactional, and partner-facing exchanges such as an API in front of the ERP or EDI with a trading partner are a different shape again, and are built through our Azure Logic Apps consulting services.
It fits organizations whose numbers already exist but never arrive in one place on time: a nightly export somebody runs by hand, a spreadsheet stitched from three systems, a warehouse that is fed by a scheduled task on a machine under a desk. It is scoped for small and mid-size organizations — one fixed-fee project with a stated end date, not an open-ended data programme.
It is deliberately not an infrastructure engagement. Moving servers and databases into Azure, and the cost work that follows, is a different job with a different scope; that is our Microsoft Azure consulting service, and the two are often sequenced with the platform work first and the pipelines after. If your reporting estate is heading toward Microsoft Fabric, pipelines built here carry the same concepts across, and our Power BI to Microsoft Fabric migration services cover that move. Where the load logic you want replaced currently lives in a third-party desktop ETL tool rather than in a scheduled task, our Alteryx to Microsoft Fabric migration services rebuild those workflows as Fabric dataflows and pipelines and reconcile every output against the original. For the reasoning behind fixed fees before you speak to anyone, how our fixed-fee process works sets out scoping, delivery, and what happens after handover.
The table shows scope and typical timeline rather than prices, because an honest number requires seeing your source list and refresh expectations first.
| Phase | Scope | Typical timeline | Fee |
|---|---|---|---|
| Phase 1 — connect and land | Data factory and staging area set up in your subscription, integration runtimes installed and registered, credentials moved into Key Vault, and the first copy pipelines running end to end against real sources | About the first week | Fixed fee — scoped after your free consultation |
| Phase 2 — shape and schedule | Transformation activities that turn staged data into model-ready tables, incremental loads with watermarks, dependency-aware triggers, parameterization so new tables are configuration, and reconciliation against the source systems | Weeks 2–3 | Fixed fee — scoped after your free consultation |
| Phase 3 — monitor and hand over | Retry rules and failure alerting, Git integration with a tested development-to-production publish, run-book documentation, an access and security review, and a walkthrough with the people who will operate it | Week 4 and close-out | Fixed fee — scoped after your free consultation |
Everything produced is yours from the first day: the pipelines, the staging schema, the deployment templates, and the Git repository all live in your own Azure subscription and your own tenant, never in ours. Azure consumption is billed by Microsoft directly to you, separately from our fixed fee.
Related services
Fixed-fee report and semantic model development for the reporting layer your pipelines feed.
Moving servers and databases into Azure and tuning what they cost — the platform work underneath.
Republishing RDL reports, rebuilding subscriptions, and retiring an on-premises report server.
Further reading
Hand-written guides from our article library.
The symptoms that show manual exports and stitching are now costing more than they save.
Why we scope in writing before starting, and what that changes about who carries the risk.
What to ask, what to check, and the answers that should make you walk away.
FAQ
You get working pipelines in your own Azure subscription: linked services to each source, parameterized copy and transformation activities, a schedule or event trigger per pipeline, failure alerts, and written run-book documentation. Everything is published from source control, so the pipelines can be changed later without guesswork.
Yes. A self-hosted integration runtime is installed on a machine inside your network and makes an outbound connection to Azure, so an on-premises SQL Server or file share can be read without opening inbound firewall ports. We install it, register it, and document how to patch it.
Power BI refreshes a model; Data Factory decides what that model reads. We land source data on a schedule, shape it into the tables the semantic model expects, and trigger the refresh afterwards, so a report is never refreshed against a load that failed halfway through last night.
They have two honest paths. Packages worth keeping can run on an Azure-SSIS integration runtime largely as they are, which suits a large estate you do not want to rewrite. Packages that are thin wrappers around a few queries are usually cheaper to rebuild as native pipelines. We recommend per package.
You do, directly to Microsoft, in your own subscription. Data Factory is billed on activity runs and data flow compute rather than a license, so the running cost belongs to you and stays visible in your Azure bill. Our fee is separate, fixed, and agreed in writing before work starts.
Typical delivery is two to four weeks from kickoff, with the date stated in your written scope. The fixed fee reflects how many sources are connected, how much transformation logic is involved, and whether an on-premises runtime is needed. Nothing starts until you approve that scope in writing.
Book a free consultation
Tell us what you’re trying to fix — a report, an approval process, an intranet, a Copilot rollout. We scope it as a fixed-fee project, you approve, and a senior engineer delivers in 2–4 weeks.
The fastest way to reach us is the form — tell us the task and we’ll reply within one business day.