SQL Smarts

  • Blog
  •  About Niall
  • Talks & Events
  • LinkedIn
  • Twitter

Azure

Azure Databricks Spark Uncategorized

Associative Grouping using Spark (Part 3)

This is part of series of posts about associative grouping: Part 1 – Associative Grouping using tSQL Recursive CTE’s Part 2 – Associative Grouping using tSQL Graph In the first two parts of this series we looked at how we could use recursive CTE’s and SQL Server’s graph functionality to Read more…

By Niall, 1 yearSeptember 30, 2021 ago
Azure AzureDataPipelineTools Data Factory Data Lake

Introducing AzureDataPipelineTools

A few months ago my friend Richard Swinbank posted a blog, More Get Metadata in ADF, about the limitations of using the Get Metadata activity in ADF to get information about files in a data lake. This to a twitter conversation as a bunch of other data engineers had been Read more…

By Niall, 1 yearSeptember 17, 2021 ago
Azure Data Factory Uncategorized

Azure Data Factory: Making Non-Dynamic Linked Services Dynamic

One of my clients has been adding data from multiple Salesforce instances to their data platform this week. One of their developers asked me if the Salesforce linked service could be made dynamic, as there is no place in the GUI to add parameters, or a dynamic values for the URL, user name or credentials. Here’s how we can do that.

By Niall, 2 yearsMay 12, 2021 ago
Azure Data Factory

Azure Data Factory Lookup: First Row Only & Empty Result Sets

When using the lookup activity in Azure Data Factory V2 (ADFv2), we have the option to retrieve either a multiple rows into an array, or just the first row of the result set by ticking a box in the UI. This allows us to either use the lookup as a Read more…

By Niall, 4 yearsMarch 23, 2019 ago
Recent Posts
  • Associative Grouping using Spark (Part 3)
  • Introducing AzureDataPipelineTools
  • Azure Data Factory: Dev Mode vs Published Code
  • Azure DevOps: SqlPackage Deployment Timeouts
  • Azure Data Factory: Making Non-Dynamic Linked Services Dynamic
Tags
ADFv2 (4) ADLSv2 (1) Associative Grouping (3) Azure (4) Azure Data Lake (1) AzureDataPipelineTools (1) Azure Devops (2) Azure Functions (1) Certificates (1) CTE (1) Databricks (1) Data Cleansing (3) Data Factory (4) Design Patterns (1) Graph DB (1) Security (1) Signing (1) Spark (1) tSQL (4) vNext (1)
  • Blog
  • About Niall
  • Talks & Events
  • LinkedIn
  • Twitter
Hestia | Developed by ThemeIsle