search

cinecert / asdcplib

AS-DCP and AS-02 File Access Library
Other
73 stars 55 forks source link

Potential bug in TimedText_Parser.cpp under WIN32 when adding subtitle assets #2

Open steveq61 opened 5 years ago

steveq61 commented 5 years ago

Hey John, I'm wondering if the below (potential) bug has been fixed in this release? Having a quick look at the source, I can't see anything different from the previous version. Cheers, Steve Q.

If I try and wrap a smpte timed text XML file which contains a font (or any other asset for that matter), the wrap process fails, as font file is not found. Asdcplib returns this error:

result {value=-9 label="The requested file does not exist on the system." symbol="NOT_FOUND" ...} Kumu::Result_t

Looking at the code for the LocalFilenameResolver function (below) in TimedText_Parser.cpp, it appears that on a Windows system, the variable found_list is being declared, but not initialised. I have defined KM_WIN32 so the code in the ifdef is ignored. Therefore, PathList_t found_list; is being declared, and the very next line is being checked for size, which will of course be 0. Consequently, it will not find my font file.

ASDCP::TimedText::LocalFilenameResolver::ResolveRID(const byte_t* uuid, TimedText::FrameBuffer& FrameBuf) const
{
  Result_t result = RESULT_NOT_FOUND;
  char buf[64];
  UUID RID(uuid);
  PathList_t found_list;

#ifndef KM_WIN32
  // TODO, fix this for win32 (needs regex)
  FindInPath(PathMatchRegex(RID.EncodeHex(buf, 64)), m_Dirname, found_list);
#endif

  if ( found_list.size() == 1 )
    {
      FileReader Reader;
      DefaultLogSink().Debug("Retrieving resource %s from file %s\n", buf, found_list.front().c_str());

      result = Reader.OpenRead(found_list.front().c_str());

      if ( KM_SUCCESS(result) )
      {
        ui32_t read_count, read_size = Reader.Size();
        result = FrameBuf.Capacity(read_size);

        if ( KM_SUCCESS(result) )
          result = Reader.Read(FrameBuf.Data(), read_size, &read_count);

        if ( KM_SUCCESS(result) )
          FrameBuf.Size(read_count);
      }
    }
  else if ( ! found_list.empty() )
    {
      DefaultLogSink().Error("More than one file in %s matches %s.\n", m_Dirname.c_str(), buf);
      result = RESULT_RAW_FORMAT;
    }

  return result;
}
jhursty commented 5 years ago

Yes, a win32 solution is needed for PathMatchRegex(), and in the mean time the error could be better explained. This is not a priority for me at the moment but having done the lift of getting onto github it is now easy for me to field pull requests.

steveq61 commented 5 years ago

Thanks John! :-)

steveq61 commented 4 years ago

Hey John, I hope you are well! I was wondering if this issue has been addressed yet please? Looking at the latest release code (2.10.35), it doesn't appear so, but I thought I'd check. Currently Windows machines can't create SMPTE subtitles that contain assets such as font files. Thanks heaps, Steve Q. :-)

stephw999 commented 4 years ago

Hi Steve ,

this is my fast solution to fix it : (TimedText_Parser.cpp) tell us if its works.

Stéphane.W

/*

THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*! \file    AS_DCP_TimedText.cpp
    \version $Id$       
    \brief   AS-DCP library, PCM essence reader and writer implementation
*/
#include "AS_DCP_internal.h"
#include "S12MTimecode.h"
#include "KM_xml.h"

using namespace Kumu;
using namespace ASDCP;

using Kumu::DefaultLogSink;

const char* c_dcst_namespace_name = "http://www.smpte-ra.org/schemas/428-7/2010/DCST";
std::string filename22 = "/";

//------------------------------------------------------------------------------------------

ASDCP::TimedText::LocalFilenameResolver::LocalFilenameResolver() {}
ASDCP::TimedText::LocalFilenameResolver::~LocalFilenameResolver() {}

//
Result_t
ASDCP::TimedText::LocalFilenameResolver::OpenRead(const std::string& dirname)
{
  if ( PathIsDirectory(dirname) )
    {
      m_Dirname = dirname;
      return RESULT_OK;
    }

  DefaultLogSink().Error("Path '%s' is not a directory, defaulting to '.'\n", dirname.c_str());
  m_Dirname = ".";
  return RESULT_FALSE;
}

//
Result_t
ASDCP::TimedText::LocalFilenameResolver::ResolveRID(const byte_t* uuid, TimedText::FrameBuffer& FrameBuf) const
{
  Result_t result = RESULT_NOT_FOUND;
  char buf[64];
  UUID RID(uuid);
  PathList_t found_list;

#ifndef KM_WIN32
  // TODO, fix this for win32 (needs regex)
  FindInPath(PathMatchRegex(RID.EncodeHex(buf, 64)), m_Dirname, found_list);
#endif

  //found_list. = m_Dirname + "/" + RID.EncodeHex(buf, 64);

      std::string filename = m_Dirname + "/" + RID.EncodeHex(buf, 64);
      filename22 = RID.EncodeHex(buf, 64);
      FileReader Reader;
      DefaultLogSink().Debug("retrieving resource %s from file %s\n", buf, filename.c_str());

      result = Reader.OpenRead(filename.c_str());

      if ( KM_SUCCESS(result) )
    {
      ui32_t read_count, read_size = Reader.Size();
      result = FrameBuf.Capacity(read_size);

      if ( KM_SUCCESS(result) )
        result = Reader.Read(FrameBuf.Data(), read_size, &read_count);

      if ( KM_SUCCESS(result) )
        FrameBuf.Size(read_count);
    }

  if ( ! found_list.empty() )
    {
      DefaultLogSink().Error("More than one file in %s matches %s.\n", m_Dirname.c_str(), buf);
      result = RESULT_RAW_FORMAT;
    }

  return result;
}

//------------------------------------------------------------------------------------------

typedef std::map<Kumu::UUID, TimedText::MIMEType_t> ResourceTypeMap_t;

class ASDCP::TimedText::DCSubtitleParser::h__SubtitleParser
{
  XMLElement  m_Root;
  ResourceTypeMap_t m_ResourceTypes;
  Result_t OpenRead();

  ASDCP_NO_COPY_CONSTRUCT(h__SubtitleParser);

public:
  std::string m_Filename;
  std::string m_XMLDoc;
  TimedTextDescriptor  m_TDesc;
  mem_ptr<LocalFilenameResolver> m_DefaultResolver;

  h__SubtitleParser() : m_Root("**ParserRoot**")
  {
    memset(&m_TDesc.AssetID, 0, UUIDlen);
  }

  ~h__SubtitleParser() {}

  TimedText::IResourceResolver* GetDefaultResolver()
  {
    if ( m_DefaultResolver.empty() )
      {
    m_DefaultResolver = new LocalFilenameResolver();
    m_DefaultResolver->OpenRead(PathDirname(m_Filename));
      }

    return m_DefaultResolver;
  }

  Result_t OpenRead(const std::string& filename);
  Result_t OpenRead(const std::string& xml_doc, const std::string& filename);
  Result_t ReadAncillaryResource(const byte_t* uuid, FrameBuffer& FrameBuf, const IResourceResolver& Resolver) const;
};

//
bool
get_UUID_from_element(XMLElement* Element, UUID& ID)
{
  assert(Element);
  const char* p = Element->GetBody().c_str();

  if ( strncmp(p, "urn:uuid:", 9) == 0 )
    {
      p += 9;
    }

  return ID.DecodeHex(p);
}

//
bool
get_UUID_from_child_element(const char* name, XMLElement* Parent, UUID& outID)
{
  assert(name);
  assert(Parent);
  XMLElement* Child = Parent->GetChildWithName(name);

  if ( Child == 0 )
    {
      return false;
    }

  return get_UUID_from_element(Child, outID);
}

//
Result_t
ASDCP::TimedText::DCSubtitleParser::h__SubtitleParser::OpenRead(const std::string& filename)
{
  Result_t result = ReadFileIntoString(filename, m_XMLDoc);

  if ( KM_SUCCESS(result) )
    result = OpenRead();

  m_Filename = filename;
  return result;
}

//
Result_t
ASDCP::TimedText::DCSubtitleParser::h__SubtitleParser::OpenRead(const std::string& xml_doc, const std::string& filename)
{
  m_XMLDoc = xml_doc;

  if ( filename.empty() )
    {
      m_Filename = "<string>";
    }
  else
    {
      m_Filename = filename;
    }

  return OpenRead();
}

//
Result_t
ASDCP::TimedText::DCSubtitleParser::h__SubtitleParser::OpenRead()
{
  if ( ! m_Root.ParseString(m_XMLDoc) )
    return RESULT_FORMAT;

  m_TDesc.EncodingName = "UTF-8"; // the XML parser demands UTF-8
  m_TDesc.ResourceList.clear();
  m_TDesc.ContainerDuration = 0;
  const XMLNamespace* ns = m_Root.Namespace();

  if ( ns == 0 )
    {
      DefaultLogSink(). Warn("Document has no namespace name, assuming \"%s\".\n", c_dcst_namespace_name);
      m_TDesc.NamespaceName = c_dcst_namespace_name;
    }
  else
    {
      m_TDesc.NamespaceName = ns->Name();
    }

  UUID DocID;
  if ( ! get_UUID_from_child_element("Id", &m_Root, DocID) )
    {
      DefaultLogSink(). Error("Id element missing from input document.\n");
      return RESULT_FORMAT;
    }

  memcpy(m_TDesc.AssetID, DocID.Value(), DocID.Size());
  XMLElement* EditRate = m_Root.GetChildWithName("EditRate");

  if ( EditRate == 0 )
    {
      DefaultLogSink().Error("EditRate element missing from input document.\n");
      return RESULT_FORMAT;
    }

  if ( ! DecodeRational(EditRate->GetBody().c_str(), m_TDesc.EditRate) )
    {
      DefaultLogSink().Error("Error decoding edit rate value: \"%s\"\n", EditRate->GetBody().c_str());
      return RESULT_FORMAT;
    }

  if ( m_TDesc.EditRate != EditRate_23_98
       && m_TDesc.EditRate != EditRate_24
       && m_TDesc.EditRate != EditRate_25
       && m_TDesc.EditRate != EditRate_30
       && m_TDesc.EditRate != EditRate_48
       && m_TDesc.EditRate != EditRate_50
       && m_TDesc.EditRate != EditRate_60
       && m_TDesc.EditRate != EditRate_96
       && m_TDesc.EditRate != EditRate_100
       && m_TDesc.EditRate != EditRate_120
       && m_TDesc.EditRate != EditRate_192
       && m_TDesc.EditRate != EditRate_200
       && m_TDesc.EditRate != EditRate_240 )
    {
      DefaultLogSink(). Error("Unexpected EditRate: %d/%d\n",
                  m_TDesc.EditRate.Numerator, m_TDesc.EditRate.Denominator);
      return RESULT_FORMAT;
    }

  // list of fonts
  ElementList FontList;
  m_Root.GetChildrenWithName("LoadFont", FontList);

  for ( Elem_i i = FontList.begin(); i != FontList.end(); i++ )
    {
      UUID AssetID;
      if ( ! get_UUID_from_element(*i, AssetID) )
    {
      DefaultLogSink(). Error("LoadFont element does not contain a urn:uuid value as expected.\n");
      return RESULT_FORMAT;
    }

      TimedTextResourceDescriptor TmpResource;
      memcpy(TmpResource.ResourceID, AssetID.Value(), UUIDlen);
      TmpResource.Type = MT_OPENTYPE;
      m_TDesc.ResourceList.push_back(TmpResource);
      m_ResourceTypes.insert(ResourceTypeMap_t::value_type(UUID(TmpResource.ResourceID), MT_OPENTYPE));
    }

  // list of images
  ElementList ImageList;
  m_Root.GetChildrenWithName("Image", ImageList);
  std::set<Kumu::UUID> visited_items;

  for ( Elem_i i = ImageList.begin(); i != ImageList.end(); i++ )
    {
      UUID AssetID;
      if ( ! get_UUID_from_element(*i, AssetID) )
    {
      DefaultLogSink(). Error("Image element does not contain a urn:uuid value as expected.\n");
      return RESULT_FORMAT;
    }

      if ( visited_items.find(AssetID) == visited_items.end() )
    {
      TimedTextResourceDescriptor TmpResource;
      memcpy(TmpResource.ResourceID, AssetID.Value(), UUIDlen);
      TmpResource.Type = MT_PNG;
      m_TDesc.ResourceList.push_back(TmpResource);
      m_ResourceTypes.insert(ResourceTypeMap_t::value_type(UUID(TmpResource.ResourceID), MT_PNG));
      visited_items.insert(AssetID);
    }
    }

  // Calculate the timeline duration.
  // This is a little ugly because the last element in the file is not necessarily
  // the last instance to be displayed, e.g., element n and element n-1 may have the
  // same start time but n-1 may have a greater duration making it the last to be seen.
  // We must scan the list to accumulate the latest TimeOut value.
  ElementList InstanceList;
  ElementList::const_iterator ei;
  ui32_t end_count = 0;

  m_Root.GetChildrenWithName("Subtitle", InstanceList);

  if ( InstanceList.empty() )
    {
      DefaultLogSink(). Error("XML document contains no Subtitle elements.\n");
      return RESULT_FORMAT;
    }

  // assumes edit rate is constrained above
  ui32_t TCFrameRate = ( m_TDesc.EditRate == EditRate_23_98  ) ? 24 : m_TDesc.EditRate.Numerator;

  S12MTimecode beginTC;
  beginTC.SetFPS(TCFrameRate);
  XMLElement* StartTime = m_Root.GetChildWithName("StartTime");

  if ( StartTime != 0 )
    beginTC.DecodeString(StartTime->GetBody());

  for ( ei = InstanceList.begin(); ei != InstanceList.end(); ei++ )
    {
      S12MTimecode tmpTC((*ei)->GetAttrWithName("TimeOut"), TCFrameRate);
      if ( end_count < tmpTC.GetFrames() )
    end_count = tmpTC.GetFrames();
    }

  if ( end_count <= beginTC.GetFrames() )
    {
      DefaultLogSink(). Error("Timed Text file has zero-length timeline.\n");
      return RESULT_FORMAT;
    }

  m_TDesc.ContainerDuration = end_count - beginTC.GetFrames();

  return RESULT_OK;
}

//
Result_t
ASDCP::TimedText::DCSubtitleParser::h__SubtitleParser::ReadAncillaryResource(const byte_t* uuid, FrameBuffer& FrameBuf,
                                         const IResourceResolver& Resolver) const
{
  FrameBuf.AssetID(uuid);
  UUID TmpID(uuid);
  char buf[64];

  ResourceTypeMap_t::const_iterator rmi = m_ResourceTypes.find(TmpID);

  if ( rmi == m_ResourceTypes.end() )
    {
      DefaultLogSink().Error("Unknown ancillary resource id: %s\n", TmpID.EncodeHex(buf, 64));
      return RESULT_RANGE;
    }

  Result_t result = Resolver.ResolveRID(uuid, FrameBuf);

  std::string resourceType;
  if ( (*rmi).second == MT_PNG )
    resourceType = "image/png";
  else if ( (*rmi).second == MT_OPENTYPE )
    resourceType = "application/x-font-opentype";
  else
    resourceType = "application/octet-stream";

  if ( KM_SUCCESS(result) )
    {
      FrameBuf.MIMEType(resourceType);
    }
  else
    {
      DefaultLogSink().Error("Resource not found: %s (%s)\n", TmpID.EncodeHex(buf, 64), filename22.c_str());
    }

  return result;
}

//------------------------------------------------------------------------------------------

ASDCP::TimedText::DCSubtitleParser::DCSubtitleParser()
{
}

ASDCP::TimedText::DCSubtitleParser::~DCSubtitleParser()
{
}

// Opens the stream for reading, parses enough data to provide a complete
// set of stream metadata for the MXFWriter below.
ASDCP::Result_t
ASDCP::TimedText::DCSubtitleParser::OpenRead(const std::string& filename) const
{
  const_cast<ASDCP::TimedText::DCSubtitleParser*>(this)->m_Parser = new h__SubtitleParser;

  Result_t result = m_Parser->OpenRead(filename);

  if ( ASDCP_FAILURE(result) )
    const_cast<ASDCP::TimedText::DCSubtitleParser*>(this)->m_Parser = 0;

  return result;
}

// Parses an XML document to provide a complete set of stream metadata for the MXFWriter below.
Result_t
ASDCP::TimedText::DCSubtitleParser::OpenRead(const std::string& xml_doc, const std::string& filename) const
{
  const_cast<ASDCP::TimedText::DCSubtitleParser*>(this)->m_Parser = new h__SubtitleParser;

  Result_t result = m_Parser->OpenRead(xml_doc, filename);

  if ( ASDCP_FAILURE(result) )
    const_cast<ASDCP::TimedText::DCSubtitleParser*>(this)->m_Parser = 0;

  return result;
}

//
ASDCP::Result_t
ASDCP::TimedText::DCSubtitleParser::FillTimedTextDescriptor(TimedTextDescriptor& TDesc) const
{
  if ( m_Parser.empty() )
    return RESULT_INIT;

  TDesc = m_Parser->m_TDesc;
  return RESULT_OK;
}

// Reads the complete Timed Text Resource into the given string.
ASDCP::Result_t
ASDCP::TimedText::DCSubtitleParser::ReadTimedTextResource(std::string& s) const
{
  if ( m_Parser.empty() )
    return RESULT_INIT;

  s = m_Parser->m_XMLDoc;
  return RESULT_OK;
}

//
ASDCP::Result_t
ASDCP::TimedText::DCSubtitleParser::ReadAncillaryResource(const byte_t* uuid, FrameBuffer& FrameBuf,
                              const IResourceResolver* Resolver) const
{
  if ( m_Parser.empty() )
    return RESULT_INIT;

  if ( Resolver == 0 )
    Resolver = m_Parser->GetDefaultResolver();

  return m_Parser->ReadAncillaryResource(uuid, FrameBuf, *Resolver);
}

//
// end AS_DCP_TimedTextParser.cpp
//
steveq61 commented 4 years ago

Hi @stephw999 thanks so much. I'll give this a run ASAP!

:-)

stephw999 commented 4 years ago

ok. :-)

I forgot to say that it is necessary to replace in the path "\" by "/". Example:

asdcp-wrap -L -a c0a325c7-6532-491e-b994-bbe2a7da4a68 "C:/Users/stephane/test/smpte-sub/subfile.xml" "C:/Users/stephane/test/outmxf/sub-test.mxf"

note: in my example there are two files in the "test\smpte-sub" directory: subfile.xml (the xml file) 313eed77-c625-47b1-ac71-3f09abc3974a (the font file)

steveq61 commented 4 years ago

@stephw999, I build asdcp.lib as a part of my DCP package and then make library calls to do the wrapping etc.. I don't use the executables (asdcp-wrap) etc, So the "/" won't be an issue. Thanks for the heads up anyhow! :-)

steveq61 commented 4 years ago

@stephw999 Hi! I have just updated to the latest stable release and included your modified code. This now appears to be working. Whilst it's not been tested on a cinema server, all indications are that it is correct. Thanks so much!