{"id":1672,"date":"2014-05-09T14:09:04","date_gmt":"2014-05-09T13:09:04","guid":{"rendered":"http:\/\/www.stuartroberts.net\/?p=1672"},"modified":"2022-09-09T13:42:11","modified_gmt":"2022-09-09T12:42:11","slug":"sending-large-data-csom","status":"publish","type":"post","link":"https:\/\/www.stuartroberts.net\/index.php\/2014\/05\/09\/sending-large-data-csom\/","title":{"rendered":"Sending large data with CSOM"},"content":{"rendered":"\n

As a follow up to my recent post showing how to create or overwrite a document using the CSOM, I’m now going to show how to upload a file containing large amounts of data. To see the original post, click here<\/a>.<\/p>\n\n\n\n

Previously, when saving the binary data, I demonstrated the functionality with the following code:<\/p>\n\n\n

\npublic void SaveFile(Microsoft.SharePoint.Client.ClientContext context, string folderRelativeUrl, string relativeItemUrl, byte[] fileData)\n{\n    var fci = new FileCreationInformation\n    {\n        Url = relativeItemUrl,\n        Content = fileData,\n        Overwrite = true\n    };\n\n    Microsoft.SharePoint.Client.Folder folder = context.Web.GetFolderByServerRelativeUrl(folderRelativeUrl);\n    Microsoft.SharePoint.Client.FileCollection files = folder.Files;\n    Microsoft.SharePoint.Client.File file = files.Add(fci);\n\n    context.Load(files);\n    context.Load(file);\n    context.ExecuteQuery();\n}\n\n<\/pre><\/div>\n\n\n
The above code, by default, will be restricts to transferring files that are no more than 2091752 bytes in size, or 2MB.

To get around this, we’ll set the content for the FileCreationInformation<\/a> object using the ContentStream property instead of the Content property.<\/p>\n\n\n\n\n\n\n\n
As it sounds, this property accepts a Stream object instead of a byte array.<\/p>\n\n\n\n
We’ll also add some logic to send the file data in buffered chunks, as there’s still a limit to how much the FileCreationInformation upload can handle in one process without error. For this, I’ve set a maximum file size of 4Mb before the buffer logic kicks in.<\/p>\n\n\n\n
The updated code looks like:<\/p>\n\n\n
\npublic void SaveFile(Microsoft.SharePoint.Client.ClientContext context, string folderRelativeUrl, string relativeItemUrl, string filename)\n{\n\t\/\/ Get the files object for the parent folder we want to upload a file to\n\tMicrosoft.SharePoint.Client.Folder folder = context.Web.GetFolderByServerRelativeUrl(folderRelativeUrl);\n\tMicrosoft.SharePoint.Client.FileCollection files = folder.Files;\n\tcontext.Load(files);\n\tcontext.ExecuteQuery();\n\n\t\/\/ 4Mb file length that will trigger a buffered upload\n\tint bufferFileSize = 4 * 1024 * 1024;\n\tlong fileLength = new FileInfo(filename).Length;\n\n\tif (fileLength <= bufferFileSize)\n\t{\n\t\t\/\/ No need to buffer the upload\n\t\tusing (FileStream stream = new FileStream(filename, FileMode.Open))\n\t\t{\n\t\t\tvar fci = new FileCreationInformation\n\t\t\t{\n\t\t\t\tUrl = relativeItemUrl,\n\t\t\t\tContentStream = stream,\n\t\t\t\tOverwrite = true\n\t\t\t};\n\t\t\t\n\t\t\tMicrosoft.SharePoint.Client.File file = files.Add(fci);\n\t\t\t\n\t\t\tcontext.Load(file);\n\t\t\tcontext.ExecuteQuery();\n\t\t}\n\t}\n\telse\n\t{\n\t\t\/\/ 2Mb buffer chunks\n\t\tint blockSize = 2 * 1024 * 1024;\n\t\tMicrosoft.SharePoint.Client.File file = null;\n\t\tGuid uploadId = Guid.NewGuid();\n\n\t\tusing (FileStream stream = new FileStream(filename, FileMode.Open))\n\t\t{\n\t\t\tbyte[] buffer = new byte[blockSize];\n\t\t\tlong offset = 0;\n\t\t\tlong totalBytesRead = 0;\n\t\t\tint bytesRead;\n\n\t\t\t\/\/ Read data from file system in blocks.\n\t\t\twhile ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)\n\t\t\t{\n\t\t\t\tClientResult<long> totalBytesSent;\n\t\t\t\ttotalBytesRead += bytesRead;\n\t\t\t\t\n\t\t\t\t\/\/ First set of data to upload\n\t\t\t\tif (file == null)\n\t\t\t\t{\n\t\t\t\t\tusing (MemoryStream emptyStream = new MemoryStream())\n\t\t\t\t\t{\n\t\t\t\t\t\tvar fci = new FileCreationInformation\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tUrl = relativeItemUrl,\n\t\t\t\t\t\t\tContentStream = emptyStream,\n\t\t\t\t\t\t\tOverwrite = true\n\t\t\t\t\t\t};\n\n\t\t\t\t\t\t\/\/ Create an empty file\n\t\t\t\t\t\tfile = files.Add(fci);\n\n\t\t\t\t\t\t\/\/ Start the initial upload of data to the file\n\t\t\t\t\t\tusing (MemoryStream bufferStream = new MemoryStream(buffer))\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\ttotalBytesSent = file.StartUpload(uploadId, bufferStream);\n\t\t\t\t\t\t\tcontext.ExecuteQuery();\n\t\t\t\t\t\t\toffset = totalBytesSent.Value;\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\t\/\/ Continue uploading this set of data\n\t\t\t\telse if (totalBytesRead < fileLength)\n\t\t\t\t{\n\t\t\t\t\tusing (MemoryStream bufferStream = new MemoryStream(buffer))\n\t\t\t\t\t{\n\t\t\t\t\t\ttotalBytesSent = file.ContinueUpload(uploadId, offset, bufferStream);\n\t\t\t\t\t\tcontext.ExecuteQuery();\n\t\t\t\t\t\toffset = totalBytesSent.Value;\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\t\/\/ We've reach the last buffered set of data to upload\n\t\t\t\telse\n\t\t\t\t{\n\t\t\t\t\t\/\/ Create byte with the final byte count\n\t\t\t\t\tvar finalBuffer = new byte[bytesRead];\n\t\t\t\t\tArray.Copy(buffer, 0, finalBuffer, 0, bytesRead);\n\n\t\t\t\t\t\/\/ Complete the file upload\n\t\t\t\t\tusing (MemoryStream bufferStream = new MemoryStream(finalBuffer))\n\t\t\t\t\t{\n\t\t\t\t\t\tfile = file.FinishUpload(uploadId, offset, bufferStream);\n\t\t\t\t\t\tcontext.Load(file);\n\t\t\t\t\t\tcontext.ExecuteQuery();\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n<\/pre><\/div>\n\n\n
This will result in the file creation either via buffered streaming or in one pass.<\/p>\n\n\n\n
Hope this helps!<\/p>\n","protected":false},"excerpt":{"rendered":"
As a follow up to my recent post showing how to create or overwrite a document using the CSOM, I’m now going to show how to upload a file containing large amounts of data. To see the original post, click … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[67,3],"tags":[91,81],"jetpack_publicize_connections":[],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/plx2I-qY","_links":{"self":[{"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/posts\/1672"}],"collection":[{"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/comments?post=1672"}],"version-history":[{"count":3,"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/posts\/1672\/revisions"}],"predecessor-version":[{"id":1962,"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/posts\/1672\/revisions\/1962"}],"wp:attachment":[{"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/media?parent=1672"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/categories?post=1672"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.stuartroberts.net\/index.php\/wp-json\/wp\/v2\/tags?post=1672"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}