PDF Day France will be the first French-speaking event of the PDF Association, organised by our member ORPALIS. It will take place in Toulouse which is the home ground of Airbus and we are very happy that Airbus will present a case study around its usage of PDF in their document management environment!Electronic Document Conference: Call for Papers
Prospective presenters at the Electronic Document Conference 2019 are invited to submit high-quality original proposals for 25-minute presentations on subjects of interest to developers and technical product managers concerned with electronic document implementations.Have we passed ‘peak PDF’?
How do we gain insight into how users’ views of documents are shifting? Google Trends is an increasingly interesting source of high-level marketplace data. By aggregating Google’s search data over time, reporting a term’s popularity as compared with all other searches.Participating in the PDF Techniques Accessibility Summit
The PDF Techniques Accessibility Summit’s objective is to establish a broad-based understanding of how PDF files should be tagged for accessibilty. It’s an opportunity to focus on establishing a common set of examples of accessible PDF content, and identify best-practice when tagging difficult cases.Members supporting PDF features!
The typical adoption curve for PDF technologies until approximately 2007 tended to track with that of the original PDF developer. Since then the marketplace has shifted; it’s no longer clear that Adobe drivesPDF feature support worldwide. Accordingly, we are happy to report that adoption of PDF 2.0 continues apace, with new vendors announcing their support every month.
PDF documents represent the bulk of newer content stored in ECM systems and cloud storage worldwide. PDF is replacing TIFF as the output format of choice for imaging systems, and theres simply no substitute for PDF when it comes to creating a shareable record of static electronic content.
Documents are often used in lawsuits and similar expensive activities. In these cases, the ability to reliably search for keywords and phrases, personally identifiable information (PII), trade-secrets or any sort of sensitive material is fundamental to understanding your risks.
Consider the following examples:
Discovery: During the document discovery process that precedes litigation, text-extraction and search processes are used to locate responsive documents, i.e., those that must be turned over to opposing counsel. Failure to turn over a responsive document can create numerous expensive problems if discovered later in the proceedings. Attorneys like to know what they’ve got.
FOIA: Freedom of Information Act procedures includes searches of extracted text to speed the identification and removal of sensitive information ranging from PII to classified data. Even so, FOIA requests are a typical channel for data breaches, as this otherwise unremarkable example makes clear.
Most of todays text-extraction tools have serious limitations when it comes to supporting reliable searching of PDF files because they dont actually account for the document they simply provide the ability to search the file. Theres a huge difference, as Ill explain.
At its heart, the PDF file-format is all about painting things onto pages, also known as “rendering”. Theres far more to PDF, of course, but thats the core of it. As a result, PDF documents dont always work precisely as one might expect when they are used for purposes other than rendering… such as searching.
Lets look at one common problem. It requires a brief dip into the guts of PDF, but not too far, and its illustrative.
PDF is extremely flexible. Its commonplace (and sometimes essential) for software to create PDF files in which the ordering of text in the file (the “content order”) differs from the texts appearance on the page (the “logical reading order”).
In Example 1, the content order is portrayed to the left. The orange arrow shows the order in which the text occurs in the file itself, which obviously differs from the way a human reads the page.
When reading the page visually, the logical reading order is what matters. When extracting text or searching, however, if the software uses the content order (how the PDF file is made) instead of the logical reading order (what’s displayed on the page), your results may be affected.
Using most currently-available tools, including Adobe Acrobat, other PDF viewers and the server-side text extraction and search features in ECM systems, searching this PDF for final check will not find that phrase even though it’s plainly present on the page.
The reason is straight-forward: the text extraction and/or search software is using the file’s content order, not the logical reading order.
Example 2 shows a document that’s visually identical to the previous example. In this case the content order matches the human readers expectations. Accordingly, a content-order search for final check or “final check list” will work.
Both PDF documents look identical on-screen and in-print but the guts of the files are quite different.
Few organizations can control the technical details of the documents they receive or create. PDF documents can and will include variations in the way text is encoded into the file, and sometimes these variations affect searchability.
If reviewers and other downstream procedures rely on text-extraction or search that uses only the content order, some hits will be missed.
To address this problem ECM customers and others who need to trust their text extraction and search systems can take the following steps:
Text extraction and search software developers have many choices about how they index PDF files. It’s up to ECM and other customers to demand software that does the right thing…. or risk the reality that some searches will fail to find each instance.
Although often defeating simpler text-ordering problems, reducing PDF documents to images in order to OCR them for will introduce new errors and cause more text to be missed in many relatively common circumstances:
Comparing text-extraction from PDF vs. OCR is easy enough to check for yourself. Take an arbitrary collection of PDF documents, extract all the text and count the words that result. Next, convert the PDFs to images and run OCR. The OCR process will (unless all pages in the example were from simple business documents) find significantly fewer words.
Today, both desktop and server-side ECM text extraction and search technology typically rests on assumptions rooted in the era of TIFF images and plain-text.
To manage exposure to the release of PII, company secrets, national security data and many other types of sensitive information, ECM customers who rely on PDF for electronic records should strongly consider reviewing the ability of their ECM systems to capture and search text from PDF documents.