AngularJS File Upload Corrupting Files

How quickly I could troubleshoot an issue if only I knew ahead of time what the cause of the problem was…

Setup: We are using AngularJS, Bootstrap, and MVC Web API REST Services to build a large web app.  This web app has several pages that provide the user with a way to upload files.  On the back-end we’re using EF 6.0 and SQL 2012 and a number of FileShares (a new feature of SQL 2012 that’s pretty cool).  We found several File Upload JS/JQuery projects online and decided to use this one:  https://github.com/danialfarid/angular-file-upload

Problem: The files being saved to the SQL FileShare were corrupt.  They wouldn’t open.  To troubleshoot this I tried a number of things:

• First, I assumed it was the SQL FileShare corrupting the files I was saving.  But it was easy to test out the FileShare and SQL wasn’t the culprit.
• Next, I assumed it was the SQL Stored Procedure I was calling that was mucking things up.  Because EF chokes on FileShare tables, we are forced to use stored procedures to upload files programmaticly into a SQL FileShare.  But I was able to verify that the stored procedure wasn’t causing the corruption.
• Next, I thought maybe it was the hand-off from the REST service to my back-end EF helper classes, but I was able to verify that the file was corrupted as soon as the REST service got it.
• At this point, I threw up my arms in confusion… what is corrupting the files I’m uploading?

It turns out, it was the Angular File Upload.  How did I determine that?  I noticed (while troubleshooting) that the files being uploaded and the file after it reached my REST service were the same size, but different sizes “on-disk”.  I thought this was worth investigating so I downloaded FrHed (http://frhed.sourceforge.net/en/) and looked at the two files side-by-side.  I found that the Angular File Upload appended a header and a footer to the image it was uploading.  What?  Why the F#$k would the file uploader append anything to the file itself?  I looked all over DanialFarid’s site for a way to disable this, but couldn’t find anything.

(Check out that header —— Webkit…. Content-Disposition….  The actual unmodified file started at YeYa..2FIF)

I had a decision to make at that point, do I try and find another upload utility or do I just fix it by hacking off the header and footer?  I went with the latter option as I thought it might be fun.  I’ve never modified an image/document like this, so it was a potential learning experience.

Solution: I’m not too happy about this solution, but it worked and it’s still working.  To get rid of the header, I noticed that there are four dots in the hex editor after “image/jpeg”.  Those are actually carriage-return-new-lines (hex 0d 0a 0d 0a).  And regardless of the image/file/document/etc… there were always those 4 dots on each file just before the original file started.  I decided to hack the header off there.  To get rid of the footer, I noticed that it was always 46 bytes long (once again, regardless of file type).  So I now had a definite beginning and ending to the file that I hoped I could programmatically build.  And it turned out that I could, but I made a bunch of mistakes along the way.

My first mistake was trying to copy the stream into a string.  The issue with this was encoding.  I could easily mess with the string using string.IndexOf and Substring, but converting the file back into a Stream was difficult due to encoding.  No matter how I encoded the string, the file remained corrupted.

Next I decided to leave the stream as a byte array and iterate through each byte looking for the sequence 0D, 0A, 0D, 0A.  This… actually worked.  Here is the code:

public static byte[] BuildByteArrayFromFileStream(System.IO.Stream obj)
{

// Removes the header introduced by AngularJS File upload. Very custom to implementation.
// Suggest changing to different upload package at some point in the future.
List<byte> byteArray = null;
using (var memoryStream = new MemoryStream())
{

obj.CopyTo(memoryStream);
byteArray = memoryStream.ToArray().ToList();

}
var header = 0;
// Static Footer added to each file uploaded is 46 bytes in length.
var footer = 46;
// Determines variable header end location and removes.
for (var b = 0; b < byteArray.Count; b++)
{

if (byteArray[b].ToString() == “13”)
{

if (byteArray[b + 1].ToString() == “10”)
{

if (byteArray[b + 2].ToString() == “13”)
{

if (byteArray[b + 3].ToString() == “10”)
{

header = b + 4;
break;

}

}

}

}

}
List<byte> final = new List<byte>();
for (var b = header; b < byteArray.Count – footer; b++)
{

final.Add(byteArray[b]);

}
return final.ToArray();

}

Like I said, I’m not super proud of this.  I’m sure there are more elegant ways to iterate through a list and look for a sequence of values, but it worked and it’s still working today.  If you read this and have a better way of doing this, please post the code below.

Capt. Rochefort