Computer Applications & Quantitative Methods in Archaeology

University of Birmingham, April 10-13, 1997

"Data-In" or "Lets try it with the horse in front" 3D modeling automated but put in its place

Author:
Steve Nickerson
Designed Building Systems Ltd.
127 Kenilworth Street
Ottawa, Ontario
Canada - K1Y 3Z4

Abstract

The computer systems we have at our disposal today offer many amazing tools to sort, analyze, display and present our data. However, much less progress has been made at the other end of this process, that of data acquisition and its accurate input into the computer formats that will do the processing and prepare the reports and graphics. One problem has been that the graphic "cart" has somehow been placed in front of the data "horse" and a lot of information has found its way into a variety of proprietary CAD formats where it is difficult to search and which may not even be accessible to the next generation of software. This data may also be available in field sketches and notes (even harder to search) but often its only computerized manifestation is as a CAD file painstakingly developed from a combination of survey and hand measurements and enhanced with a great deal of assumption and generalization into a "drawing" that looks right and presents the hypothesis in the best possible light. What is needed is a data acquisition strategy that acknowledges that our primary purpose is to create a database accommodating, or at least pointing to, all the available information. One which can be easily searched.

"Data-In" or "Lets try it with the horse in front"

3D modeling automated but put in its place

Our CART is driven by the data. The strategy is to create a database of all the "objects" that make up a site, in a table coded with keywords for their fixed characteristics, and with links to other tables where their softer characteristics (those that may change with further study or future discoveries) are described. There are also links to image and text files to allow a more flexible and complete description of each object.

CART (Computer Aided Recording Tools) is the name we have given a suite of tools, both software and hardware, which tries to address the problems of data collection and accessibility by first helping to structure the on-site survey process so that more consistent data is collected, and then satisfy the perceived need for "drawings" by automatically generating a 3D CAD model from the resulting database.

This automatic drafting can save vast amounts of time and provide valuable tools for managing a project, as processing can be done on a laptop computer before the site has to be altered. However, the most significant feature is that all the field records (measurements, keywords, notes, photographs and sketches) can be integrated into an easily searchable structure to which more specialized data files, describing the different object types, can be related. Queries to any of these tables can produce a CAD model distinguished by the characteristic that each entity of the model is linked to all the tables, text files and images associated with that object, all accessible with a mouse click.

The Database

The underlying structure of these tools is a database of points similar to that shown below:
 
ANGLE  DISTANCE  DEPTH  P_NO  P_ID  O_NA  LOCUS  ISAT  PH  DATE  SZ_X  SZ_Y  SZ_Z  COMMENTS 
55.500  75.300  100.000  137.0  L  L13  13  S  25/06/95  0.0000  0.0000  0.0000  locus outline - type Soil 
56.000  204.400  105.000  138.0  L  L13  13  S  25/06/95  0.0000  0.0000  0.0000  " 
9.500  247.100  107.400  139.0  L  L13  13  S  25/06/95  0.0000  0.0000  0.0000  " 
338.500  170.300  104.900  140.0  C  L13  13  S  25/06/95  0.0000  0.0000  0.0000  close locus area 
20.000  141.200  100.900  141.0  EL  L13  13  S  25/06/95  0.0000  0.0000  0.0000  spot elevation 
4.500  92.800  95.600  142.0  EL  L13  13  S  25/06/95  0.0000  0.0000  0.0000  depths are relative but the 
32.000  185.200  104.700  143.0  EL  L13  13  S  25/06/95  0.0000  0.0000  0.0000  values printed are absolute 
348.700  146.200  116.500  144.0  FRESCO  95011200  13  S  26/06/95  12.0000  8.0000  2.0000  artifact fragments 
9.000  206.700  117.900  145.0  FRESCO  95011201  13  S  26/06/95  6.0000  4.0000  1.5000  O_NA = artifact number 
14.500  236.700  112.100  146.0  FRESCO  95011202  13  S  26/06/95  8.0000  4.5000  2.0000  for large objects 
356.500  206.000  135.000  146.1  FRESCO  95011203  13  S  28/06/95  0.5000  0.0000  0.0000  or the bag number 
344.500  185.000  136.800  146.2  FRESCO  95011203  13  S  28/06/95  0.0000  0.0000  0.0000  for small ones or groups 
79.000  43.700  146.000  154.0  L  L14  14  A  28/06/95  0.0000  0.0000  49.0000  locus 14 - Architecture 
53.000  212.300  143.000  155.0  L  L14  14  A  28/06/95  0.0000  0.0000  0.0000  has a thickness of 49cm 

Five fields are mandatory though their exact nature is configurable. Three fields define the location of the point in 3d space. This example uses angle, distance and depth as read from an omphalos (described below) but several other measurement systems are supported (including X-Y-Z, V_angle-H_angle-distance, as well as running dimensions). The P_ID (point ID) field contains a code which tells the CAD program how to deal with the point and the O_NA, (object name) is a unique name for the object being recorded. The size fields (SZ_X, SZ_Y, SZ_Z) are not required but can be used for enhancements such as giving a size to an inserted symbol or extruding a line (representing a wall for instance) to give it a height or thickness. There is also a field, P_NO (point number) which links the database to the field notes and survey data, and one for unstructured comments. The other fields are user defined repositories for keywords, values or dates. Such fields are optional and are not used by the CAD program except for things like layer naming. They would typically contain information categories such as colour, strata, locus number, excavation date, etc..

Automatic Drafting

The automatic drafting component of the system is controlled by the Point ID. As each point is processed this value tells the CAD program whether to use that point to draw a line or arc, to help define a coordinate system or to how to store it until enough information has been processed to allow some more complex parametric operation to be performed. At present the codes triggering lines, 3d faces and symbol insertions can be configured by the user while more complex structures such as 3d arcs and solids are pre-defined.

Drawing Intelligence

The basic intelligence of the model is a function of the Object Name which groups all the points that make up a particular object. The database is processed by the interpretation software in groups defined by this field and when it changes everything drawn under the previous object name is blocked and labeled with that value It is these labels that provide the connection between the generated model and the various database tables describing the objects. By default, each object is linked to both the recorders database (described above) and to an object specific table determined by the Point ID. However, any table that has this Object Name in a key field could be accessed by clicking on the object.

Generating a Model

A model is generated by sending a query to the database using SQL (Standard Query Language). It can be as simple as select * from project which would generate the entire database. Taking examples from the database fragment above the following examples will give an idea of the possibilities:

select * from project where LOCUS='13' draws only the material from locus 13

select * from project where P_ID='FRESCO' the fresco fragments from the whole site

select * from project where DATE like '%/06/%' everything excavated during June (any year)

or even select * from project where DATE like '%/95' and P_ID='FRESCO' and SZ_Z>1.5 which would insert a symbol for each fresco fragment found in '95 that was more than 1.5 cm thick.

Another possibility is to query one or more of the associated database tables to generate a model. A query such as: select * from LOCUS where MATERIAL like 'SILT%' would consult the LOCUS table, where all the loci for the site would be detailed and pass the object names of those where the MATERIAL field indicates silt (or silty, silted, etc.) To the drawing generator which would then draw each locus volume that conformed to the stated criteria. As always, each object drawn will have links to both the database tables and other associated files allowing access to a variety of information by clicking on an object, in the example above, a symbol representing a fragment of ivory.

Other Links and File Naming

As well as the obvious connection to the database that produced the model and to the other tables that support the data type, the object name is also the key to gaining access to files of other types. A code is typed or selected from the menu indicating the program/file-type desired and the user is prompted to select an object. If a file, in the format specified, exists for the object selected the necessary program is launched and the file presented for viewing and/or editing. If no file exists the user is asked if they want to create one. Used in this way the automatically generated model becomes a front end providing access to all the files and database tables associated with each object without the user needing to know the specific name of the object. Digital images and text files are directly supported by the software in its current manifestation but any program and file type could easily be added.

The naming of objects(Nickerson 1995-1) in such a way that their names can be used as filenames will allow anything to be linked to anything else and, at this simplest level, no specialized software is required to make these associations. For archival purposes of course this is the ideal as any future researcher would be able to make the connection between the different elements if they all bear the same name or keyword. If database software is available it will be much easier but the primary information can be stored in any number of formats, including hard-copy and still maintain much of its integrity.
 

Detail Drawings

 
The need for presentation drawings will always exist and the models generated automatically are unlikely to satisfy this requirement unless an extraordinary number of points are recorded. Furthermore, if this level of detail existed at all times the drawing file would often be too large to handle effectively. The way we have approached this problem is to provide a suite of functions to enable the user to create a separate drawing from the entities that have been drawn automatically. A selected object, as well as any other entities that may be required to provide context, are exported to a new drawing file and from these elements a coordinate system and several key points are available to the CAD draftsperson as a starting place for modeling the detail. Several aids to 3d drafting have been incorporated into the environment to make it easier to work in the coordinate system(s) of the object and this drawing can be enhanced to any level of detail desired. Of course the resulting file has the same name as the object name in the database and when the detail option is chosen, either for a specific object or for the whole model these drawings will be inserted in place of their schematic counterparts.
 

Layer Names

The primary tool for sorting data in most CAD drawings is the layer name and in the interest of producing a drawing file that maintains at least some of the information contained in the associated database tables we have developed a methodology that encodes multiple database fields into a complex layer name that will, with the aid of a custom menu item allow certain pre-defined queries from the user. Database keywords may indicate locus, object type, excavation date, material, etc. and these values are incorporated into a layer name that will enable the entities conforming to the specified criteria to be hidden, exposed or coloured as determined by the user. While the use and meaning of these layer names will not be intuitively obvious to the uninitiated a table on a readme layer of the drawing can keep the essential information with the drawing.

Data Storage and Archiving

A first principle of any heritage record should be that the data should Sunday, November 16, 1997 be as universally accessible as possible(Nickerson 1996-2). This means that, as far as it is possible, the ability to read the computer files should not be dependent on any proprietary software (including our own). Thus the archival resting place of the geometric and keyword information should not be an Autocad or other CAD format but as a .DBF database file, a format that is readable by almost every database or spread sheet program available today (and, as it happens, by Autocad as well). Carried to its logical conclusion this policy would suggest that these data files be written out in a comma or tab delimited format and saved as ASCII text or even printed out. In this case researchers in the future could reconstruct the site manually based on the coordinate and other information, even if they had no computer facilities at all. Alternatively they could write an interpreter for their current software just as we have done for Autocad. CARTs geometry and database files are in this format by default though the data access engine will support various other formats (including client/server). Its text files are, of course, stored as ASCII text and raster image files, though configurable, default to the .GIF format not because this is some common standard but, as there is no standard, their relatively small file size and their widespread use on the Internet lends them some credibility.

Data Acquisition

Of course what you do with the data is the easy part of the problem, many useful tools are available for getting the data out of a well structured database or file system (several were presented at this conference). Building the database is a much more difficult and less studied problem. CART is based on the assumption that a single table should be available that is capable of pointing to all the data even though there will be a variety of formats that will be used to store and display it (computer files, hard copy, video and even the artifact itself all need to be found) . Flexibility is the key but as long as unique object names are used (and how could you find anything without at least this) anything bearing that name in whole or in part can be located - easily by the software and with only slightly less facility by a thoughtful researcher.

There are, however, certain file formats that are more appropriate for certain types of data than they are for others. For instance, our fixation on the "drawing" (a legacy from a pre-digital era) has forced much information onto large pieces of paper that would be better served by an annotated photograph, digital image or simple text file. The migration of the different types of information from the field or the object to the database will be different depending on the nature of the information. Coordinates can come from photogrammetry, hand measurements or survey, images from photos, sketches, video or digital cameras and text from notes, laptop computer or data collector. Though the final format is known the path from field to archive may take several twists and turns. The accompanying flow chart (Nickerson 1994,1996-1) shows one such

scenario where the primary intent was for quick (same or next day) data update. Here, with the exception of the film photography everything could be processed on site or at the site office and the resulting database could be used for the day to day management of the dig (Nickerson 1996-3). Digital cameras and the faxing of sketches (from hotel lobby to a laptop modem) kept the images up to date. Though these are often of a less than optimal quality they can be replaced by proper scans whenever appropriate facilities became available.

Measurement Tools

Though the coordinate information can come from virtually any source, from scaling a paper drawing, to extractions from existing CAD drawings, to a state of the art total station survey instrument we have, for the most part, been using the following devices.

For situations where a large number of points can be collected quickly we have adopted a modified surveying system which uses a visible (and reflectorless) EDM coupled with an electronic theodolite. This combination allows fast collection of points (4-5 per minute in good conditions) and levels of accuracy more than adequate for the objects in question.


For the slower work of excavation we have adopted the omphalos which consists of a compass rose, permanently fixed at a point in or near the square. It supports a tape measure in a cradle that allows relatively quick measurements using in a polar-cylindrical coordinate system (elevations are obtained by the use of a rod and bubble level attached to the tape)(Nickerson 1995-2,1996-3).

Conclusions

CART is first and foremost a data acquisition strategy. Different types of information demand different file types and different projects require different approaches to data acquisition but in all cases the final product must be a searchable database that provides at least enough information to identify the object and to locate all the available information which pertains to it. At present it can best be described as a working prototype though it has been doing real work for several years. It is available to interested parties or institutions on a contractual/beta testing basis and where it goes from here depends on what projects and requirements these partners come up with.

Bibliography

Nickerson, S. 1996-1
Report on CART, a Computer Aided Recording Tool
Automation in Construction 5 (1996) 161-170
Elsevier Science Publishers
http://nickerson.icomos.org/steve/elsevier
Nickerson, S. 1996-2
The (non-virtual) Reality of the Heritage Record
Proceedings of the ICOMOS 11th General Assembly
Sofia, Bulgaria - October 1996
http://nickerson.icomos.org/steve/current/sofia.htm
Nickerson, S. 1996-3
Recording Humeima '95
Scholars Press, Atlanta Ga, USA Religious Studies News 11(2):4-23
http://nickerson.icomos.org/steve/hum95/hum95.htm
Nickerson, S. 1995-1
Object Oriented Recording
http://nickerson.icomos.org/steve/oor.htm
Nickerson, S. 1995-2
A Comparison of Measurement Techniques in Archaeology and Heritage Recording http://nickerson.icomos.org/steve/measure.htm
Nickerson, S. 1994
A Site Information System (SIS): CADD/Database Integration for Field Use.
APT Bulletin 24(1):56-62
http://nickerson.icomos.org/steve/sis.txt
More up to date information concerning the CART project may be found at: http://nickerson.icomos.org/steve/