Load tables

From SFU_Public
Jump to: navigation, search

Preliminary Notes                                       (emonos page)

Each menu may list a) its use; b) how it matches; c) overlay action; and d) which fields are protected and how.

The Protection List is in two sections: "F" for fixed-length field section, and "V" for variable-length field section.

To check what the numbers or characters represent, go to Create Lists and start a search on bibliographic records, Type b. Double-click in the "Field" box to see the list of possible fields. Beside each field is either its constant number (for fixed-length fields) or its field tag character (for variable-length fields). Protection for variable-length field tags may specify MARC tags, represented either literally or by regular expression. In REGEX, the dot represents any character.

Protected MARC tags should also be qualified by a protection code (either d or k). These codes define behaviour for when the defined MARC tag is present in both records: the new one (or incoming record) and the existing one (or target record). Basically there are 3 conditional protection possibilities at the MARC tag level :

  • code d = delete from incoming record (and keep this tag if present in target record)
  • code k = keep from incoming record (and delete this tag if present in target record)
  • no code (default) = keep tag from incoming record (and keep this tag in target record) < results in duplicate fields after loading.

If only the variable-length field tag is defined along with codes d or k, the behaviour is flawed (condition only applies to the first MARC tag of any group). To counter this flaw, always use REGEX to define the MARC tag, e.g. a(...:d)b(...:d) etc.

When neither code d or code k is used, the behaviour is usually undesired as it results in either duplicate fields or data loss, i.e.:

  • field/MARC tags present in protection list = duplicate fields (all MARC tags loading to the variable-length field are protected, and all load from incoming record).
  • field/MARC tags absent from protection list = deletion of target fields (applies to all MARC tags loading to the variable-length field: none are protected, all load from incoming record).

For example: Protection list: F28-31V78bc(09.:d)

Means: protect fixed-length fields 28 through 31; protect variable-length fields 7 and 8; protect all b-tagged MARC tags and load all incoming. Protect all existing c-tagged 09X fields and delete all incoming; conversely: delete those not in the 09X range, and load all incoming. It also means: delete all e-tagged MARC in the existing record, regardless of whether an e-tagged MARC is present in the new incoming record. Likewise, delete all other valid tags (f, g, h, i, etc) and load all incoming if present, since these tags are not included in the protection list either.

For more details on protection and its codes, see the User Manual at Page # 101871.

List of Current load menu items on X:\CataloguingDivision\Downloads\Load Menu.xlsx   1407

List of all load tables in a spreadsheet on X:\CataloguingDivision\Downloads\loaders.xlsx   - 2012

Locally-controlled menus in Data Exchange


A - Generic e-book - (m2btab.ebook) - used for loading most ebooks, e.g. books24x7, etc.

   Used for e-books for which we can't get update files, but only the whole file each time, so overlay action is set to reject if match is found (1407ps)


Matches on
Overlay Action
Record Changes
Use
001
reject
Location=web



MatType= m



Add cat date



533 - if present take out |n



655 - if present delete



856 - add proxy pre-pend



add canned 594

NOTE:  776 not loading 0905, fixed by Frances.

mapping: 856 - add proxy pre-pend; also add canned 594 [and 538 - commented out march 2, 2011ps].

594 changed to new licensing note 13/10/02ps

Template used: netbib

Protection list: F26,28-31V178b(:d)c(0..:d)d(6..:d)e(:d)f(:d)g(:d)h(59.,850:d)j(930:d)m(035,935)n(505,535:d)q(776,777:k)s(4..,8..:d)u(:d)w(:d)x(:d)y(035,690:d)z(:d)

B -  DO NOT USE - hardcoded in Troy as B > REMOVE files; changing this will disrupt nightly scripted routines (FD, July 2014).


C - CRC bib file (m2btab.engnet) updated  1312ps

Matches on URL in 935
creates bibs only
used for Engnet (CRC) files

mapping: 856 - add proxy pre-pend; add canned 594 and cat date, changes 010|a to 010|z to accomodate matching by records from OCLC Promptcat

Template used: bksebk

Protects the series statement in 830, 935 and 035


D - LOAD electronic orders (m2btab.erm)

Load rush orders from the ERM system.

Alternate menu: Electronic monograph orders (m2btab.couttspda)

For ebrary monos, and formerly, Coutts PDA.

MATCHES on 001 or 037
location =Web;
III grid mat type = m;
overlay action: attach
No cat date
adds:

594 Licensed for 1 simultaneous user at Simon Fraser University.

Templates used: bksebk, eorder

Creates orders from 980/981


E - loadoclc (m2btab.boclc)

For batchloading bibs only from OCLC Connexion (or individual bibs and items)

MATCHES on 001
may also be used to match on Millennium .b (or other) system number from 949 in command line
creates bibs only
overlay action: insert, overlay, and insert if more than one match
Template used: full

Protection list: F26V1678c(099)d(690)g(086)hj(930:d)n(505,520,538,535,594,595)y(856,887)

Note: table duplicates the one in Load MARC OCLC BIB records (ocl)

G - Load & match batch orders (m2btab.bip2)

May use for initial batch-load of individual orders.

Same as H but matches to deduplicate on ISBN.


H - LOAD Title-by-Title ebook orders & upgrades (m2btab.bip)

Originally created to download records from BIP, since used to bring in upgraded records from Connexion.

Now used to load title-by-title orders and associated upgrades.

Matches on 001 or m-tagged 935

Templates used: bksebk, eorder

Most fields are overwritten, per protection list: F26-31V178j(930:d)m(035)

I - Springer bibs (m2btab.springer)* (still used for OCLC overlay of Springer files 1407ps)

MATCHES on persistent URL redirected to m-tagged 935

               [two maps in the springer table:

                       1) to copy part of the 856 and redirect the copy to indexed 935. The redirect starts after "http://" as this was determined the only common point in the URL, for the various springer files.
                       2) to insert proxy to the 856.]
c) may also be used to match on Millennium .b (or other) system number from 949 in command line
creates bibs only
overlay action: overlay if one match is found; insert otherwise
Protection: Most fields are overwritten or deleted: standard fixed-length fields and MARC tags 930 and 856 are conditionally protected:

Protection list: F26,28-31Vj(930:d)y(856:k) added r(199:d) to use when overlaying Naxos records with OCLC records. 150213ps

used to download records from Springer and OCLC

mapping: 856 - add proxy pre-pend; add canned 506, 538, and cat date.

Template used: netbib

  • alternate table (m2btab.spring) was created for one-time load to match on ISBN with a confirmation on title, and only loads the 856. Protection: all variable-length fields are overwritten.
  • 594 changed to new licensing note 13/10/02ps


J - CJDB (m2btab.ejdb)

matches on Millennium .b (or other) system number in 035
also matches on m-tagged (g-indexed) 035
creates bibs only
overlay action: insert, overlay, reject if more than one match
used to create and overlay records from CJDB

Protection list: b=F24,26-31,81V78a(1..:d)b(7..:d)c(099:d)d(6..:d)e(2..:d)f(0..:d)g(:d)h(59.:k)h(850:d)i(02.:d)j(930:d)k(:d)l(01.:d)m(935:d)n(3..,5..:d)o(001:d)

p(2..:d)q(7..:d)r(3..:d)s(..0:d)t(:k)u(130,246,730,740:d)v(:d)w(58.,787:d)x(780:d)y(00.,222,04.:d)y(856:k)z(785:d)

K - Thesis file (m2btab.theses) (still used 1407ps)

MATCHES bibs from Millennium .b (or other) system number in 035 |a
creates bibs and items from 966 and 967
overlay action: insert, overlay, insert if more than one match
used for ongoing thesis loads

Templates used: thesesbiblod,thesesitem

Protection list: F28-31V78a(:d)b(700,71.:d)c(099:d)d(6..:d)e(:d)f(0..:d)g(:d)h(59.,850:d)i(:d)j(930:d)l(:d)mn(500,506,516,530,533,538,546,555:d)o(:d)p(:d)q(:k)r(:d)s(4..,8..:d)t(:d)u(:d)w(:d)x(:d)y(00.,035,040,094,690,856:d)z(:d)

L - OCLC Bib/Item (m2btab.bioclc)

For batchloading bibs and items from OCLC Connexion

MATCHES on Millennium .b (or other) system number in 035
creates bibs and items (items created from 949 second indicator 1)
overlay action: insert, overlay, insert if more than one match
cat date.

Templates used: biblod, tapei

Protection list: b=V178c(099:d)d(690:d)g(086:d)hj(930:d)n(505,520,538,535,594,595,598:d)y(094,856,887:d)

M - Aux Amateur bib/order (m2btab.amateursord)

MATCHES on ISBN with confirmation on title as indexed
creates bibs and orders (status on-order)
overlay action: insert, attach, insert if more than one match
used to load bibs and orders from Aux Amateurs de Livres

Templates used: couttsbib,amateurord

Protection list: F26,28-31V178bc(09.:d)d(6..:d)efgj(930:d)h(59.,850:d)m(035)n(505,535:d)oqs(4..,8..:d)uwxy(035,690,856:d)z


N - Load an ebook file with overlay (m2btab.eboverlay)

Same as load menu A (m2btab.ebook) with overlay action=o and utf8 turned on.


O - Load OCLC Holdings xref files (m2btab.xref)

MATCHES on .b number in 035

OCLC number must be in 935
Overlay action: attaches the OCLC number in 035|s addint (OCoLC)


R - CRDB file (m2btab.ermdb)

MATCHES on j-tagged m-indexed fields (ERM number)

overlay action: insert, overlay, reject if more than one match

used to load CRDB bibs from ERM

Following tags are mapped and overlay any in existing records:

245 > u290
246 > u290
500 > n592
961h > h590
856 > y856

Template used: CRDB
Other MARC tags in the existing record are protected from overlay, per protection list:

F24,26,28,29,89V_178a(:d)b(:d)c(09.:d)d(6..:d)e(:d)f(05.,09.:d)g(:d)h(590,591,592,850:k)i(02.:d)j(930:d)l(:d)m(035,920,935:d)n(505,520,53.:d)n(59.:k)o(001:d)p(:d)q(:k)r(3..:d)s(4..,8..:d)t(245:d)u(210,246,299,730,740:d)w(:d)x(:d)y(0..,222,035,690:d)z(:d)"


Alternate table, m2btab.erm, loads ERM order records:

MATCHES bibs from Millennium .b (or other) system number in 907 and 908
also MATCHES on j-tagged m-indexed fields, or on m-indexed fields
overlay action: insert, overlay, insert if more than one match
used to load bibs and orders (status 1) from ERM

Templates used: ERM,erm

Protection list:

F26,28-31V78a(:d)b(:d)c(09.:d)d(6..:d)e(:d)f(05.,09.:d)g(:d)h(59.,850:d)i(02.:d)j(930:d)l(:d)m(035:k)n(505,535,59.:d)o(001:d)p(:d)q(:k)r(3..:d)s(4..,8..:d)t(:d)u(210,246,299,730,740:d)w(:d)x(:d)y(0..,222,690,856:d)z(:d)

'

S - Barcodes file (m2btab.barcode)

MATCHES items from Millennium system number in 035
If match fails, then secondary match is on the barcode
overlay action: none
used to match known items and save to review file


T - Springer reject file (Still used for load of Springer records, so as not to overly OCLC records 1407ps)

Copied Springer changed only the overlay action to 'r' so that overlays will be rejected.  This is to protect the OCLC records we have loaded over the Springer records.

MATCHES on persistent URL redirected to m-tagged 935

[two maps in the springer table:

1) to copy part of the 856 and redirect the copy to indexed 935. The redirect starts after "http://" as this was determined the only common point in the URL, for the various springer files.
2) to insert proxy to the 856.]
c) may also be used to match on Millennium .b (or other) system number from 949 in command line
creates bibs only
overlay action: overlay if one match is found; insert otherwise
Protection: Most fields are overwritten or deleted: standard fixed-length fields and MARC tags 930 and 856 are conditionally protected:

Protection list: F26,28-31Vj(930:d)y(856:k)

used to download records from Springer

mapping: 856 - add proxy pre-pend; add canned 506, 538, and cat date.

Template used: netbib

alternate table (m2btab.spring) was created for one-time load to match on ISBN with a confirmation on title, and only loads the 856. Protection: all variable-length fields are overwritten.
594 changed to new licensing note 13/10/02ps


U - Alexander Street Press (m2btab.asp) (earlier used for:Books24x7 (m2btab.247)) Overlay on 035
May, 2016 (m2btab.review2)- changed to be able to just replace URL and leave everything else for ebrary to Deslibris conversion (also adds "830 0desLibris.|pDocuments collection")

Template used: bksebk

Overlay action:  insert, overlay, reject


W - LOAD RDA overlay from Backstage

Matches on Millennium record number in 035

  Copy of authority load table (bna)


X - Load bibs to a review file (m2btab.review) (Still used 1407ps)

used to match known bibs and save to review file MATCHES on 'o' overlay action: reject / attach / reject

Can be changed to MATCH bibs from Millennium system number in 035 (035  $a.bnnnnn)
Secondary match is on o-index or m-tagged g-indexed 037 with a confirmation on edition
overlay action: u


Y - LOAD e-records, no proxy added (m2btab.marcive) (Still used 1407ps)

MATCHES bibs from Millennium .b (or other) system number in 035
also matches on 001
creates bibs only
(once used to load bibs from Alexander Street Press)

Now re-purposed to load free electronic resources, e.g. gov docs from Marcive, BC Legislative Library, etc.; also used for ebrary records.

overlay action: insert, overlay, insert if more than one match

Protection list: b=F26,28-31V178b(:d)c(09.:d)d(6..:d)e(:d)f(:d)g(:d)h(59.,850:d)j(930:d)l(:d)m(035)n(505,535:d)o(:d)q(:k)s(4..,8..:d)u(299:d)w(:d)x(:d)y(035,690,856:d)z(:d)

Template used: bksebk


Z - Serials recon (m2btab.serialscl)

MATCHES bibs from Millennium system number in 939, with a confirmation on first 4 characters of the normalized 245

overlay action: insert, overlay, insert if more than one match

Template used: full

Protection list: V0123456789h(590,591,592:d)i(022:k)jn(535:d)u(299:d)y(856:d)F26,28,31

Tables not listed in Data Exchange Local Menu


Z39.50 Remote searching table (m2btab.b)

Matches on bib as selected by user

Template used: _b

Protection list: F26,31V678c(099)f(050,055,090)h(59.)j(930:d)m(935)n(5..)u(290:d)s(830:d)y(856,962)


Invoice-loader (m2btab.pcat)

Matches on order record number

Secondary match on LCCN and OCLC # for approvals

Templates used: bksapp,loadapp

Protection list: F28-31V78b(710)c(099:d)e(:d)g(:d)h(59.,850:d)m(935)m(035:k)n(535,594:d)q(:k)s(4..,8..:d)u(:d)w(:d)x(:d)y(006,007,035,690,856:d)z(:d)

Overlay protection hierarchy: |||0|0| | |0|n|G|0|@ov_priority=" :1:I:4:2:L:M:5:8:7:u:3:z:K:0:J"
Used to load invoices and upgrade vendor records from Coutts Oasis (PromptCat) and Aux Amateurs de Livres.


Table used for authority updates (m2btab.bna)

Matches on Millennium record number in 035

overlay action: insert, overlay, insert if more than one match

Template used: full

Protection list: V0123456789c(099:d)h(59.,850:d)j(930:d)m(035)n(535,505:d)p(260:d)r(300:d)t(245:d)u(246,247,299,740:d)w(580,787:d)x(780:d)y(008,035,043,856:d)z(785:d)F26,28,31

NOTE:  added 594-595 to field list (was 590-593) - 23ja12ps


Table used for Coutts Oasis orders (m2btab.couttsorder)

Matches on ISBN

overlay action: insert, attach 856, or insert if more than one match

Templates used: couttsbib,coutordlod

Protection list:

F28-31V_78b(:d)c(09.:d)d(6..:d)e(:d)f(:d)g(:d)h(59.,850:d)j(930:d)m(:d)n(505,535:d)o(:d)p(:d)q(:d)r(:d)s(4..,8..:d)t(:d)u(:d)w(:d)x(:d)y(856)y(035,690:d)z(:d)"
@ov_attach_insert="y(856)n(594)  -- inserts 856 and 594 into existing bib record.

@recs="couttsorder" -- name of load table
@dflt="couttsbib,coutordlod --  templates assigned

@pre_map="m2bpremap.coutts" -- maps data to Millennium.


Tables fallen to disuse:

m2btab.bookhouse





-el flag to see errors

In the m.marcload.local as follows:

adjust this line from:
P > pPREPROCESS records loaded via FTS||marccopy -u -i%#.lfts -o%%.lmarc
to
P > pPREPROCESS records loaded via FTS||marccopy -u -eal -i%#.lfts -o%%.lmarc

b) uncomment this line:
#V > vview errors ||review -c %#.errs