Found a significant error, please fix heroes!

May 20, 2010 at 11:52 PM
Edited May 24, 2010 at 1:58 AM

System.IO.FileFormatException: File contains corrupted data.
   at MS.Internal.IO.Zip.ZipIOEndOfCentralDirectoryBlock.FindPosition(Stream archiveStream)
   at MS.Internal.IO.Zip.ZipIOEndOfCentralDirectoryBlock.SeekableLoad(ZipIOBlockManager blockManager)
   at MS.Internal.IO.Zip.ZipArchive..ctor(Stream archiveStream, FileMode mode, FileAccess access, Boolean streaming, Boolean ownStream)
   at MS.Internal.IO.Zip.ZipArchive.OpenOnStream(Stream stream, FileMode mode, FileAccess access, Boolean streaming)
   at System.IO.Packaging.ZipPackage..ctor(Stream s, FileMode mode, FileAccess access, Boolean streaming)
   at System.IO.Packaging.Package.Open(Stream stream, FileMode packageMode, FileAccess packageAccess, Boolean streaming)
   at DocumentFormat.OpenXml.Packaging.OpenXmlPackage.OpenCore(Stream stream, Boolean readWriteMode)
   at DocumentFormat.OpenXml.Packaging.WordprocessingDocument.Open(Stream stream, Boolean isEditable, OpenSettings openSettings)
   at DocumentFormat.OpenXml.Packaging.WordprocessingDocument.Open(Stream stream, Boolean isEditable)
   at fleXdoc.Api.DocBuilder.Build(XPathNavigator dataNavigator, String nsPrefix, String ns, Stream output, CultureInfo renderCulture, Boolean validatePackage) in G:\Ondemand\BS_Dev\Infrastructure\ReportEngineV2\FlexDoc\DocBuilder.cs:line 293
   at fleXdoc.Api.DocBuilder.Build(Stream data, String nsPrefix, String ns, Stream output, CultureInfo renderCulture, Boolean validatePackage) in G:\Ondemand\BS_Dev\Infrastructure\ReportEngineV2\FlexDoc\DocBuilder.cs:line 251

Error here:

-----------------------------------------------------------

 WordprocessingDocument doc = WordprocessingDocument.Open(docIn, true); //line 293
                using (fleXdocTemplateProcessor processor = new fleXdocTemplateProcessor())
                {
                    if (ResourceUriResolve != null) // If our event is subscribed, then we also need to subscribe to fleXdoc's event
                    {
                        processor.ResourceUriResolve += delegate(object sender, ResolveResourceUriEventArgs args)
                        {
                            return ResourceUriResolve(sender, args); // Just pass through
                        };
                    }
                    processor.ProcessWordprocessingDocument(doc, options);
                }

----------------------------------------------

this Error sometimes appears, sometimes does not appear,At the same time generate 100 word document,word2007 template with the xml data in exactly the same,Why???

thank very much!

flystudio

2010/05/21

 

Coordinator
May 21, 2010 at 8:04 AM
Edited May 21, 2010 at 8:05 AM

Hello flystudio,

I think I know what's the cause of the error. Check DocBuilder line 280:

 

 

 

// Create an in-memory working-copy of the template
// TODO: Add locking
using (MemoryStream docIn = IOHelper.LoadIntoMemoryStream(_template)) 
{

 

As you can see there's still a TODO there. Since the code only copies data in memory, chances of running into this issue are minimal, as I've not run into them when I was stresstesting fleXdoc myself (just 10 concurrent requests). I will fix this as soon as I can find the time. Up until then, you'll need to modify your own copy of the code. Solution is not that hard, just add locking to the _template variable.

Good luck,
Robert

Coordinator
May 21, 2010 at 8:17 AM

Ok, couldn't stand having such a serious issue in fleXdoc, so I fixed it anyway :)

Can you check to see if this fix really resolves your issue?

May 21, 2010 at 9:01 AM

modify:

   //using (MemoryStream docIn = IOHelper.LoadIntoMemoryStream(_template))
            //{

        to:

    MemoryStream docIn = _template;

client:

    MemoryStream doc = new MemoryStream();
                using (DocBuilder target = new DocBuilder(template))
                {
                    using (Stream data = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(reportXmlData)))
                    {
                        string nsPrefix = "o";
                        string ns = ""; //"http://dataschemes.com/data";
                        target.Build(data, nsPrefix, ns, (Stream)doc, CultureInfo.CurrentCulture, false);

                        // Basic checks
                        // Write the outputfile to disk (for manual inspection)
                        using (Stream fileOutput = File.Open(wordName, FileMode.Create, FileAccess.ReadWrite))
                        {
                            IOHelper.CopyStream(doc, fileOutput);
                            fileOutput.Flush();
                            fileOutput.Close();
                        }
                    }
                }

 this Problem still exist...

HELP!!

Coordinator
May 21, 2010 at 9:13 AM

I use LoadIntoMemoryStream to create a clone of the template. I do so to make sure the OOXML SDK will never modify the stream containing the original template. During this cloning process another thread can attempt to read the same stream. Since the stream has a current position, etc, you don't want to allow access from multiple threads. That's why a added locking to the code.

Did you check out my modification already? I really think it should fix your problem!

May 21, 2010 at 9:18 AM

 //using (MemoryStream docIn = IOHelper.LoadIntoMemoryStream(_template))
            //{

or

MemoryStream docIn = _template;

 Problem still exist...

 

 

May 21, 2010 at 9:27 AM

this Error sometimes appears, sometimes does not appear,100 in 3-5 this error, sometimes a mistake not, very depressed。

Coordinator
May 21, 2010 at 9:29 AM

Dude, did you even check out my modification to the source???

May 21, 2010 at 9:49 AM
Edited May 21, 2010 at 10:14 AM

     

     #region Private instance fields
        private MemoryStream _template;
        private object _templateLock = new object();
        private bool _disposed = false;
        #endregion

------------------------------------------

.......

MemoryStream docIn = null;
            lock (_templateLock)
            {
                docIn = IOHelper.LoadIntoMemoryStream(_template);
            }

            // Create an in-memory working-copy of the template
            // TODO: Add locking
            using (docIn)
            {

                // Process the working-copy using the Open XML SDK
                BuildOptions options = new BuildOptions();
                options.Data = dataNavigator;
                options.NamespacePrefix = nsPrefix;
                options.NamespaceUri = ns;
                options.RenderCulture = renderCulture;
                options.ValidatePackage = validatePackage;
  

                WordprocessingDocument doc = WordprocessingDocument.Open(docIn, true);
                using (fleXdocTemplateProcessor processor = new fleXdocTemplateProcessor())

....

---------------------

Problem still exist...

May 21, 2010 at 10:18 AM

  use code:

 private void Build(XPathNavigator dataNavigator, string nsPrefix, string ns, Stream output, CultureInfo renderCulture, bool validatePackage)
        {
            if (dataNavigator == null)
            {
                throw new ArgumentNullException("dataNavigator");
            }
            if (output == null)
            {
                throw new ArgumentNullException("output");
            }
            if (!output.CanWrite)
            {
                throw new ArgumentException("output-stream is read only", "output");
            }
            if (renderCulture == null)
            {
                throw new ArgumentNullException("renderCulture");
            }
           
            // Create an in-memory working-copy of the template
            // TODO: Add locking
            //using (MemoryStream docIn = IOHelper.LoadIntoMemoryStream(_template))
            //{

                MemoryStream docIn = _template;
                // Process the working-copy using the Open XML SDK
                BuildOptions options = new BuildOptions();
                options.Data = dataNavigator;
                options.NamespacePrefix = nsPrefix;
                options.NamespaceUri = ns;
                options.RenderCulture = renderCulture;
                options.ValidatePackage = validatePackage;

.....

---------------------------------------------------

CCR Number of threads set to 1,Problem solving,but, if CCR Number of threads set to 10, Problem still exist...


  

May 21, 2010 at 11:56 AM
Edited May 21, 2010 at 11:57 AM

   System.IO.FileFormatException: File contains corrupted data.

  ---------------------------------------

.....

 options.Data = dataNavigator;
                options.NamespacePrefix = nsPrefix;
                options.NamespaceUri = ns;
                options.RenderCulture = renderCulture;
                options.ValidatePackage = validatePackage;
  

                WordprocessingDocument doc = WordprocessingDocument.Open(docIn, true);  //this code is OpenXmlv2.0.50727, this is OpenXml Bug!!!,now determine ,Don't support multi-threading
                using (fleXdocTemplateProcessor processor = new fleXdocTemplateProcessor())
                {
                    if (ResourceUriResolve != null) // If our event is subscribed, then we also need to subscribe to fleXdoc's event
                    {
                        processor.ResourceUriResolve += delegate(object sender, ResolveResourceUriEventArgs args)
                        {
                            return ResourceUriResolve(sender, args); // Just pass through
                        };
                    }
                    processor.ProcessWordprocessingDocument(doc, options);
                }

----------------------------------

flexdoc is great, this bug has nothing to do with the flexdoc

thank you very much robertk ,You are very professional!

 

Coordinator
May 21, 2010 at 12:06 PM

Hmm, thanks for the thumbs up! :)

Where did you find that information? This is new to me.

May 22, 2010 at 2:04 AM
To ensure thread safety? ? Please fix flexdoc Heroes. help!!!
May 22, 2010 at 3:09 PM

This problem has been solved, thanks. :)

Coordinator
May 23, 2010 at 6:17 PM

Do you mean my code-change did the job? Or did you work around the problem some other way?

May 24, 2010 at 1:50 AM
Edited May 24, 2010 at 2:58 AM

 private void Build(XPathNavigator dataNavigator, string nsPrefix, string ns, CultureInfo renderCulture, bool validatePackage, Stream template, string wordName)
        {
            if (dataNavigator == null)
            {
                throw new ArgumentNullException("dataNavigator");
            }
         
            if (renderCulture == null)
            {
                throw new ArgumentNullException("renderCulture");
            }

            // Create an in-memory working-copy of the template

            //TestTemplateStream(template);

            Stream docIn = template;

          

            using (docIn)
            {

                // Process the working-copy using the Open XML SDK
                BuildOptions options = new BuildOptions();
                options.Data = dataNavigator;
                options.NamespacePrefix = nsPrefix;
                options.NamespaceUri = ns;
                options.RenderCulture = renderCulture;
                options.ValidatePackage = validatePackage;

   
        
                WordprocessingDocument doc = WordprocessingDocument.Open(docIn, true);
                using (fleXdocTemplateProcessor processor = new fleXdocTemplateProcessor())
                {
                    if (ResourceUriResolve != null) // If our event is subscribed, then we also need to subscribe to fleXdoc's event
                    {
                        processor.ResourceUriResolve += delegate(object sender, ResolveResourceUriEventArgs args)
                        {
                            return ResourceUriResolve(sender, args); // Just pass through
                        };
                    }
                    processor.ProcessWordprocessingDocument(doc, options);
                }
                doc.Close(); // Write all changes back to the underlying stream
                docIn.Seek(0, SeekOrigin.Begin);

                using (Stream fileOutput = File.Open(wordName, FileMode.Create, FileAccess.ReadWrite))
                {
                    IOHelper.CopyStream(docIn, fileOutput);
                    fileOutput.Flush();
                    fileOutput.Close();
                }
            }
        }

 

-----------------------

 

   private static Stream GetTemplateDocStream(string cacheKey, string templateDoc)
        {
            Stream templateDocStream;

            dictionaryTemplateDocCache.TryGetValue(cacheKey, out templateDocStream);

            if (templateDocStream == null)
            {
                try
                {
                    lock (thisTemplateDocLoadLock)
                    {
                        //logger.Info("++++++++++++++++++++++Load Template from file+++++++++++++++++++++++++");
                        templateDocStream = IOHelper.GetFile(templateDoc, FileMode.Open, FileAccess.Read, FileShare.Read);
                        if (!dictionaryTemplateDocCache.ContainsKey(cacheKey))
                            dictionaryTemplateDocCache.Add(cacheKey, templateDocStream);
                    }
                    MemoryStream targetDocStream = CopyMemoryStream(templateDocStream);
                    return targetDocStream;
                }
                catch (Exception e)
                {
                    logger.HandleException(e, "ReportEngine.TemplateParse");
                    throw e;
                }
            }
            else
            {
                //logger.Info("++++++++++++++++++++++Load Template from cache+++++++++++++++++++++++++");
                MemoryStream targetDocStream = CopyMemoryStream(templateDocStream);
                return targetDocStream;

            }
          
        }

  private static MemoryStream CopyMemoryStream(Stream templateDocStream)
        {
            int initialCapacity = 32768;
            try
            {
                long templateSize = templateDocStream.Length;
                initialCapacity = Convert.ToInt32(templateSize);
            }
            catch (NotSupportedException e)
            {
                logger.HandleException(e, "ReportEngine.TemplateParse");
                throw e;
            }
            MemoryStream targetDocStream = new MemoryStream(initialCapacity);
            lock (thisTemplateStreamCopyLock)
            {
                IOHelper.CopyStream(templateDocStream, targetDocStream);
            }
            
            return targetDocStream;
        }

 

------------------------

  Stream template = GetTemplateDocStream(reportTemplate, templateDoc);


                MemoryStream doc = new MemoryStream();
                DocBuilder target = new DocBuilder();
                using (Stream data = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(reportXmlData)))
                {
                    string nsPrefix = "o";
                    string ns = ""; //"http://dataschemes.com/data";
                    target.Build(data, nsPrefix, ns, CultureInfo.CurrentCulture, true, template,wordName);

                }

May 24, 2010 at 1:52 AM

 you code:

 public DocBuilder(Stream template)
            : base()
        {
            _template = IOHelper.LoadIntoMemoryStream(template); //here need lock
        }

 

Coordinator
Jul 23, 2013 at 6:44 PM
Come check out the successor of fleXdoc: Docati!
It supports Word 2010 and 2013 as well and runs in the cloud.

I sure hope to welcome you as a Docati user as well! :-)

http://www.docati.com