Metadata Read/Write Mapping

From ResourceSpace Documentation Wiki

Jump to: navigation, search

Contents

ResourceSpace's Metadata Support with ExifTool

In ResourceSpace versions 1.5 and above, Exiftool can be used to read and write metadata. This is extremely flexible and any metadata present in a file can be mapped to ResourceSpace fields. Many different profiles are supported in addition to Exif, including XMP, IPTC, GPS, JFIF, GeoTIFF, ICC Profile, Photoshop IRB, FlashPix, AFCP and ID3.

The method is highly unique and flexible, yet quite simple in concept. It features:

  • the security of knowing that your original assets will always be left untouched, in case metadata writing corrupts a file (which can happen any time a file is modified). The metadata write happens upon download, so if the downloaded file is corrupt, the original asset is still accessible.
  • Exiftool, which efficiently reads an enormous quantity of tags from all kinds of files, and writes to many formats.
  • The ability to map any tag in a file to any ResourceSpace field.
  • The ability to map multiple tags to the same ResourceSpace field. So, for example, you have a few places to pick up a Caption depending on different files you have, and ResourceSpace will attempt to level the field by writing its extracted info to all of those tags upon download.
  • Different Resource Types can have different mapping strategies.
  • A metadata report gives a full exiftool report on the original file, showing all possible metadata that could have been indexed, sorted by metadata type (IPTC,XMP,EXIF, etc). It also shows which tags are mapped to which RS fields, and a diff report showing how any tags have been subsequently modified in ResourceSpace fields (which gives an indication of how the file will be modified on download).
  • ResourceSpace fields can be mapped to custom XMP panel tags. With a modification of the exiftool config file, it can also write to those fields.

Working with Exiftool

You must have Exiftool (a command-line interface to a PERL library) installed and enabled (the path must be specified) in config.php or auto-detected during the setup routine.

In order to map an IPTC/Exif/XMP tag name to a ResourceSpace field, you can enter ExifTool-supported tag name(s) in the "Exiftool_Field" box in Resource Types/Fields.

Here are some example mappings. Your default installation will probably have these and more already set up. :

RS Field Exiftool_field Tags
Title Title
Keywords Keywords
Caption Caption-Abstract,Description,ImageDescription
Country Category,Country
Named Persons People
Rating Rating
Credit Source,Creator,Credit,By-line
Camera Make/Model Model
Notes JobID

There are further special cases which are handled by ResourceSpace itself, not Exiftool:

Description Exiftool_field Tags
Original filename Filename
Original filename without extension StrippedFilename


See ExifTool documentation to look up tag names. It fully supports read/writing of several filetypes including JPG, GIF, PNG, TIF, EPS, PDF. It can also read data from other filetypes that are not write-supported.

http://www.sno.phy.queensu.ca/~phil/exiftool/#features

http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html <--good documentation of command-line capabilities

Download ExifTool Man Page in PDF Format

Remember that any individual field may not be available for a particular filetype. For example JobID will not write to a PDF file. If you need "Notes" to be written to PDF files, you will have to find a PDF-writable field in addition to JobID. The reason for using multiple fields is that certain programs look for information in a specific place. For example if Photoshop gets the caption from Caption-Abstract and another program you use gets it from ImageDescription, you need to have your RS caption in both. It may take some experimentation to get the combination that best suits your workflow.

Of course you can also create your own custom fields and mappings. Let's say you want a weblink to be embedded in the metadata of your images. You simply create an RS field called something like "URL" and in the Exiftool_field you write "WebStatement", which is a field you've found in ExifTool's list of writeable fields that is most appropriate for that kind of data.

Custom XMP namespaces with ExifTool and Adobe CS4/CS5

Adobe CS4/CS5 extensively utilizes XMP metadata across its applications. The XMP File Info SDK from Adobe allows you to setup custom file info panels and thus defining XMP namespaces for use with Adobe programs. ExifTool can then read these namespaces and ResourceSpace in turn is able to store these metadata in its database.

In Adobe CS4/CS5, custom file info panels are a set of XML documents defining the elements in use by the panels and some code written in Adobe's open source programming language, Flex, which is an ActionScript derivative. However, by utilizing the 'generic' panel from the XMP File Info SDK, you can quite easily create custom file info panels only by editing the two XML documents (manifest.xml and properties.xml) to accommodate basic needs, as the panel itself handles the layout of widgets and text based on the contents of the manifest.xml and properties.xml files. Along with a calendar/date widget, the 'generic' panel allows you to use text fields, text boxes, labels, check boxes and drop-down menus for your panel layout.

The manifest.xml file describes the common properties of your schema and panel while the properties.xml contains the schema and panel definitions themselves. A sample manifest.xml file is shown here: 

<xfi:fileinfo xmlns:xfi="http://ns.adobe.com/xmp/fileinfo/">
	<xfi:panels>
		<xfi:panel
			name = "ExampleXMPSchema"
			label = "Example XMP Schema"
			description = "Custom File Info Panel for use with Adobe CS5 applications. XMP Schema developed for use with ResourceSpace."  
			type = "custom"
			version = "1.0"
			panelLibrary = "generic"
			panelClass = "examples.panels.generic.GenericPanel"
			propertyDescriptionFile = "properties.xml"			
			localizationFile = "Generic"			
			modifyDate = "2011-06-15Z"
			visible="true">
		</xfi:panel>
	</xfi:panels>
</xfi:fileinfo>

A sample properties.xml file is shown here: 

<?xml version="1.0" encoding="UTF-8"?>
<xmp_definitions xmlns:ui="http://ns.adobe.com/xmp/fileinfo/ui/">
	<xmp_schema prefix="example" namespace="http://example.com/ExampleNamespace/"
		label="$$$/Custom/Schema/Label=Custom Properties" description="$$$/Custom/Schema/Description=This is my example panel.">
		<!-- simple properties -->
		<xmp_property name="ExampleText" category="external"
			label="$$$/Custom/Property/TextInputLabel=Text Field:" type="text" />
		<xmp_property name="ExampleBool" category="external"
			label="$$$/Custom/Property/BooleanDisplay_Label=Check Box Display:"
			type="boolean" />
		<xmp_property name="ExampleDate" category="internal"
			label="$$$/Custom/Property/DateDisplay_Label=Date Display:" type="date" />
		<xmp_property name="ExampleInt" category="external"
			label="$$$/Custom/Property/IntegerInputLabel=Integer Field:" type="integer"
			ui:format="{0} cm" ui:width="50%" />
		<ui:separator />
		<!--simple properties (choices) -->
		<xmp_property name='ExampleChoice' category='internal' type='closedchoice'
			label='$$$$/Custom/Property/ClosedChoiceDisplay_Label=Closed-Choice Display:'>
			<xmp_choice raw_value='1' label='$$$/xmp/example/ExampleChoice/1=1' />
			<xmp_choice raw_value='2' label='$$$/xmp/example/ExampleChoice/2=2' />
			<xmp_choice raw_value='3' label='$$$/xmp/example/ExampleChoice/3=3' />
		</xmp_property>
		<ui:separator />
		<!-- arrays -->
		<xmp_property name="ExampleBag" category="external" type="bag"
			label="$$$/Custom/Property/TextArrayInput_Label=Text Array Input:"
			element_type="text" />
		<ui:separator />
	</xmp_schema>
</xmp_definitions>

Together, these two XML-documents will render this basic file info panel:

Image:FileInfo_example.jpg

The different 'xmp_property_name' tags will correspond to your XMP metadata tags, that ExifTool will read from your files. This means that ExifTool will read the following metadata tags based on our sample schema:

  • ExampleText
  • ExampleBool
  • ExampleInt
  • ExampleDate
  • ExampleChoice
  • ExampleBag

Once mapped to fields in ResourceSpace, these values will import into ResourceSpace and if you define your custom namespace/tags in the ExifTool config file, you can also write to those fields from ResourceSpace.

To map your XMP custom schema tags and allow ExifTool to comfortably read these from your files, you will need to setup a custom schema definition in ExifTool's configuration file, .ExifTool_config which resides in the same directory as ExifTool itself (e.g. Ubuntu and Debian places this in: /usr/share/libimage-exiftool-perl/.ExifTool_config). Below is a sample .ExifTool_config file that matches the above mentioned sample schema: 

XMP-Example:ExampleText
XMP-Example:ExampleBool
XMP-Example:ExampleInt
XMP-Example:ExampleDate
XMP-Example:ExampleChoice
XMP-Example:ExampleBag

'Image::ExifTool::XMP::Main' => {
	Example => {
		SubDirectory => {
			TagTable => 'Image::ExifTool::UserDefined::Example',
		},
	},
},

%Image::ExifTool::UserDefined::Example = (
    GROUPS => { 0 => 'XMP', 1 => 'XMP-Example', 2 => 'Image' },
    NAMESPACE => { 'Example' => 'http://example.com/ExampleNamespace/' },
    WRITABLE => 'string',
    ExampleText => { },
    ExampleBool => { },
    ExampleInt => { },
    ExampleDate => { },
    ExampleChoice => { },
    Bag => { },
);

The first part of the file defines the metadata tags that are to be included in our custom namespace, the second part maps our custom namespace 'Example' to the XMP::Main namespace that is the namespace that ExifTool will present towards applications like ResourceSpace. The last piece of the file allows us to overwrite or replace standard metadata tags (i.e. Dublin core) with values from our newly created custom schema. (See Phil Harvey's documentation).

For a really well-written guide to the basics of the 'generic' custom file info panel visit Gunar Penikis' blog.

Notes:

  • As of r1456 (version numbers greater than *.*.1456) it is possible to enter the group followed by a colon then the field name, if you need to be specific to a particular metadata group (for example JFIF:XRESOLUTION).
  • As of r739, Resource Type is recognized by the ExifTool feature. This means, for example, that Video Type resources will not extract or write Photo Type or Document Type fields. Exiftool operations and reports are limited to Global Fields and Fields of the current resource type.
  • Date reading and writing of RS date fields works now, but time storage needs some more thought. Now, date is preserved but times are truncated.

Metadata Reference Materials

http://www.metadataworkinggroup.org/specs/ standards on working with metadata

http://www.adobe.com/devnet/xmp/ custom file info panels

http://www.prismstandard.org/recordedwebinars09/xmpwebinars.htm xmp presentations

Metadata Report

New feature, in development, in subversion r736, if you set $metadata_report to true, the ability to run a full metadata report is added to the View page. This can help you find useful information embedded in the original files, or help generate a mapping strategy.

It shows:

  • all metadata in the original file, by the exiftool name.
  • which fields have been mapped to RS and extracted
  • which fields have been subsequently changed in RS.
  • it also hints at the read/writability of fields.

In short, it shows any RS mapping that intersects with current embedded metadata.

It does not show RS mappings that may write *new* previously-non-existent tags to the file. It simply focuses on what existing information is being extracted and which of that info may be affected by the attempted metadata-write upon download.

Exiftool versions less than 7.47 will not give accurate feedback about the writability of particular file formats (only tags in general). v7.47 and above will allow the report to omit suggestions that tags are writable if the file itself is not writable.

Mapping Exif / IPTC without ExifTool

If you have not installed ExifTool, ResourceSpace supports the reading of some basic Exif fields and all IPTC fields when resources are uploading. For anything other than the most basic metadata support it is best to use ExifTool as described above.

Without ExifTool, ResourceSpace will not write altered metadata back to files on download.

Exif fields are mapped using the $exif_* settings in include/config.php.

IPTC fields are mapped using the 'IPTC Equiv.' option on each field in System Setup. The codes you need are as follows:

IPTC Equiv. value Metadata field extracted
2#120 Caption
2#025 Keywords
2#055 Creation Date
2#040 Special Instructions
2#085 Credit Byline Title
2#101 Country
2#105 Headline
2#110 Source
2#115 Photo Source
2#120 Caption
2#090 City
2#095 State
Retrieved from "http://wiki.resourcespace.org/index.php/Metadata_Read/Write_Mapping"
Personal tools