Using SharePoint Online (SPO) and Azure Logic Apps helps to automate processes. For a project, we have created some workflows with Azure Logic Apps that monitor changes to an SPO list or document library for approvals and other document workflows. We have found that the SPO connector does not always send all data in a reproducible manner: sometimes some fields were sent and sometimes not. We struggled with this behavior and found an unexpected workaround.
The scenario
The project consists of a simple contract management, using a (Microsoft Teams) document library in SPO. When a document is ready for the workflow, a bunch of metadata (fields) are filled out. When the workflow starts, various conditions of the item are checked in an Azure Logic App. Approvals are sent, permissions are changed, PDFs are created and so on. The document library was extended with additional fields, such as Status, Approvers, and much more. The following screenshot shows a sample of the Contracts library.
We use an Azure Logic App with the SharePoint Online Connector and the action "When an item is created or modified". The Connector checks for changes in the document library. Then, the work starts. Variables are initialized with data from the action trigger, e.g. getting the value for the field "Approver1" by getting it with @triggerBody()?['Approver1'], and so on. The following graphics shows the idea of the Logic App.
You can find out more how to connect SPO with an Azure Logic App at Monitor and manage SharePoint resources with Azure Logic Apps.
The issue
The connector is fired at the given time range and gets the item that has been changed. The run shows the trigger Inputs and Outputs, similar as here.
We saw that the connector sometimes did not deliver ALL item data of the Contracts list…
We could reproduce that strange behavior: Sometimes all fields were delivered, sometimes one or two fields were missing. They were not empty, but simply missing in the output. Therefore the flow ran into errors when we referenced to a field that was missing. We didn't find any documentation for this behavior. On the other hand, we have no influence on the behavior of the trigger action and the logic in the connector. So, this had to be solved somehow.
The workaround
We played around with this behavior but couldn't fix it first. Then we got the idea that this could be caused by any limitations in the connector. See more about the SPO connector limits here. There is a suitable paragraph here, but the following link does not lead to the solution: "If you are getting an incomplete data set or you are not able to return accurate results from the SharePoint list, it might be because of delegation limits." The same can happen if you have too many Lookup fields in the list.
After all, we suspected that the SPO list simply had too many fields (or too much content) to completely transmit an item to the Logic App. The behavior may be due to the amount of data that is transferred in the HTTP post.
So, our workaround was to create a new view - in this sample "workflowdata" - in the SPO list that delivers only the required data for the workflow to reduce the data.
The view only shows a handful - the required - columns for the Logic App. In this sample, we marked the relevant fields with the red box and the rest, the unneeded columns, in the blue box. In our case there were a few dozen columns that we didn't need in the workflow.
As last step, the Azure Logic App is now using the reduced data from the view "workflowdata".
We are not sure whether the behavior described is common. However, with this workaround, the issue is fixed.
We hope this article helps if you encounter similar issues.