During an incremental refresh of a model built on 30 CSV files in OneLake, the refresh fails due to resource exhaustion. What is a plausible cause?

Prepare for the DP-600 Fabric Analytics Engineer Exam. Study with flashcards and multiple choice questions, each offering hints and detailed explanations. Enhance your chances of success on the exam!

Multiple Choice

During an incremental refresh of a model built on 30 CSV files in OneLake, the refresh fails due to resource exhaustion. What is a plausible cause?

Explanation:
Incremental refresh works best when the system can push the filtering down to the data source, a process known as query folding. When folding is active, only the new or changed partitions are read from the 30 CSV files in OneLake, keeping memory and compute needs in check. If query folding does not occur, the engine has to pull all data from all files and perform the incremental filtering in memory, which can surge memory usage and CPU load. With 30 CSV files, this rapid, unfiltered data load can easily exhaust resources and cause the refresh to fail. So the most plausible cause is that query folding is not occurring. If the data source were offline, you’d expect a connectivity error instead of resource exhaustion. If the data weren’t incremental, the refresh behavior would differ (it might try to refresh more data), but the symptom described is most directly explained by the absence of folding. If the dataset had errors, you’d typically see data or load errors rather than a generic resource exhaustion failure.

Incremental refresh works best when the system can push the filtering down to the data source, a process known as query folding. When folding is active, only the new or changed partitions are read from the 30 CSV files in OneLake, keeping memory and compute needs in check. If query folding does not occur, the engine has to pull all data from all files and perform the incremental filtering in memory, which can surge memory usage and CPU load. With 30 CSV files, this rapid, unfiltered data load can easily exhaust resources and cause the refresh to fail. So the most plausible cause is that query folding is not occurring.

If the data source were offline, you’d expect a connectivity error instead of resource exhaustion. If the data weren’t incremental, the refresh behavior would differ (it might try to refresh more data), but the symptom described is most directly explained by the absence of folding. If the dataset had errors, you’d typically see data or load errors rather than a generic resource exhaustion failure.

Subscribe

Get the latest from Passetra

You can unsubscribe at any time. Read our privacy policy