Few Limitations of Snowflake Tasks

Snowflake TASKs are used to run one or more SQL statements or call a stored procedure. A TASK can be setup to run on a ⏰ schedule. Also, we can chain the tasks to run one after another, to form a DAG pipeline using the 'AFTER' clause.

But..., if you are relying only on Snowflake's capabilities for pipeline orchestration, there are few limitations to keep in mind.

1. Say we have a pipeline task1>task2>task3. If one of the child, task2 fails, you can't restart it using 'Execute Task task2' command.

2. The CREATE TASK statement doesn't validate the underlying SQL. i.e You can create a task to read data from a non existing table.

At this point, it is better to rely on external orchestration tools like Azure Data Factory, unless your snowflake procedures were designed bullet proof, to rerun from the root task any number of times without affecting the data integrity.

Thanks for reading.

#snowflake #task