languageTemplates(2)

FILE LANGUAGE TEMPLATES

MicroEmacs 2009 provides a large range of macros and templates to deal with the most commonly occurring types of ASCII file that may be edited. However, there is a requirement for users to extend this capability to include more obscure file types, in addition to bespoke files found internally within organizations, or devised by the user.

For each file type, MicroEmacs 2009 may be tailored to recognize the file and modify it's hilighting, key binding configuration, osd display and indentation to accommodate the file. In addition, new shorthand macros may be introduced to help deal with the contents of the file.

This section outlines the steps to be taken to integrate a new file language template into MicroEmacs 2009. buffer-init(3) automates many of the operations defined below and it is suggested that this is a preferable method to add a new language template into MicroEmacs.

The scope of the File Type

The first step is to decide the scope of the file, this will determine where the file hook should be defined. The options are:-

A standard file type not supported


If this is a standard file type not supported by MicroEmacs 2009 then it should be added to me.emf, in addition contact us and we will add it to the standard release. Any macro files associated with this file type should be available globally and are added to the MicroEmacs macro directory.

Local To your organization


If it is a file type local to your organization then it should be added to your company.emf file. Any macro files associated with the file type should be added to your local company MicroEmacs 2009 directory.

Local to an individual


If this is a file type that is only used by a limited number of individuals then it should be added to the user.emf file. Any files associated with the file type are added to your local user MicroEmacs 2009 directory.

Recognizing the File Type

The next step to adding a new file type is to get MicroEmacs 2009 to recognize the file as the new type. Recognition is performed by the File Hooks which perform recognition on the file extension and/or the file content. The name of the file type must be determined, this is typically the name of the file prepended by hk. e.g. a file with extension foo uses the file hkfoo.emf for it's language specific definitions.

Using the add-file-hook(2) invocation the file recognition is bound to the file hook macro whenever the file type is loaded. The file hook is added to the appropriate global, company or user start up file as determined in step 1. The file hooks for file foo might be defined as follows, depending upon the recognition method:-

Recognizing the extension


To recognize the file extension, then a space separated list of extensions may be defined, including the dot '.' (or other) extension separator.

add-file-hook ".foo"       fhook-foo 

Recognizing a magic editor string in the file


If the file type adopts multiple extensions (or does not use a file extension) then an editor specific string may be inserted into the file to enable the editor to recognize it, typically of the form -!- type -!-, if the string is GNU Emacs compatible then the -*- convention may be used. The binding is defined as:-

-1 add-file-hook "-!-[ \t]*foo.*-!-"           fhook-foo 

Recognizing a magic string in the file


UNIX files use a "#!<path>" notation for executable ASCII files. If the file is this type of file (or uses any other type of common string in the as the first characters of a file) then the binding may be defined as follows, in this case we have assumed foo is the UNIX executable variety i.e. #!/usr/local/bin/foo:-

1 add-file-hook "^#!/.*foo" fhook-foo 

Any, or all of the above recognition methods may be employed to invoke the language specific macro. Note that the methods are evaluated in a LIFO order, hence it is possible to over-ride an existing method.

Defining the Macro File

Once the hook has been defined, the language specific file must be created. Create the language specific file with the same name as defined in the hooks, removing the fhook- prefix and replacing it with hk, i.e. fhook-foo invokes the language specific file hkfoo.emf. Create, the file and add the file hook macro. for example hkfoo.emf contents may be defined as:

define-macro fhook-foo 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 
ml-write "[MicroEmacs foo file hook loaded]" 

The file hook may be tested by exiting and re-loading MicroEmacs 2009, or simply by executing the file containing the add-file-hook function. Once the file bindings are installed a foo file may be loaded and the hook message should be displayed.

The best way to create a new file hook is to find an existing file type with similar support to the new file hook you wish to create and re-use the hook code to create the new one.

Modifying an Existing file hook

The standard file hooks supplied with MicroEmacs 2009 should not be modified, typically a user will want to extend the repertoire of hi-lighting tokens to encompass locally defined programming libraries or syntactical extensions, in addition to extending support macros that are associated with the file type. In this case, an extension to the hook function is required. The hook file myXXX.emf, allows extensions to be made to the hkXXX.emf, without editing the original file. This may be considered to be an include file and is executed, if it exists, after the hk file has been executed. i.e. if the hook file hkfoo.emf is already defined and extensions are added to myfoo.emf.

Note that the myXXX.emf files do not typically include any fhook-XXX functions, the original fhook functions would be used. However, if a different buffer environment is required from the one created be the hook, such as a different setting of tab(2m) mode, the hook function should be copied to myXXX.emf and altered appropriately.

Most extensions and changes to the buffer environment may be performed using myXXX.emf. If the extension macro file includes a macro definition my-fhook-XXX, then my-fhookXXX is automatically executed after fhookXXX has been run.

Adding Hilighting definitions

File specific hilighting is used to pick out key words and tokens used within the file type, it greatly improves readability; the hilighting is also used for printing. The hilighting is defined within the body of the file and is executed once when the hook file is loaded, this occurs when the hook function is executed. During development of the hilighting code, it is usually necessary to execute the hook buffer to view the effects of any changes to the hilighting.

The hilighting is defined using the command hilight(2) which requires a hilighting identifier, used to identify the hilighting scheme. This identifier is dynamically allocated when the hook file is loaded, again using foo, the identifier is allocated at the top of the file and is protected such that a value is assigned once only.

!if &sequal .hilight.foo "ERROR" 
    set-variable .hilight.foo &pinc .hilight.next 1 
!endif 

The variable .hilight.next allocates unique hilighting numbers, typically a single hilighting number is consumed, incrementing the .hilight.next variable ready for the next allocation. The hilighting color scheme is defined in a macro variable .hilight.ext, where ext is the name of the language scheme (i.e. foo).

Given a hilighting number, the hilighting scheme may be defined. Each of the tokens in the language is assigned a hilighting color, for our simple foo file type:-

0 hilight .hilight.foo 1                $global-scheme 
hilight .hilight.foo 2 "#"              .scheme.comment 
hilight .hilight.foo 4 "\"" "\"" "\\"   .scheme.string 
hilight .hilight.foo 0 "'.'"            .scheme.quote 
hilight .hilight.foo 0 "'\\\\.'"        .scheme.quote ; '\?' quoted char 

hilight .hilight.foo 1 "if"             .scheme.keyword 
hilight .hilight.foo 1 "then"           .scheme.keyword 
hilight .hilight.foo 1 "else"           .scheme.keyword 
hilight .hilight.foo 1 "endif"          .scheme.keyword 

When the hilighting tokens have been defined, the hilighting scheme is bound to the buffer. This is performed by assigning $buffer-hilight(5) with the hilighting scheme within the fhook macro body, e.g.

define-macro fhook-foo 
    ; Assign the hilighting 
    set-variable $buffer-hilight .hilight.foo 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 

Putting it all together hkfoo.emf now comprises:-

!if &sequal .hilight.foo "ERROR" 
    ; Allocate a hilighting scheme number 
    set-variable .hilight.foo &pinc .hilight.next 1 
!endif 

; Define the hilighting scheme 
0 hilight .hilight.foo 1                $global-scheme 
hilight .hilight.foo 2 "#"              .scheme.comment 
hilight .hilight.foo 4 "\"" "\"" "\\"   .scheme.string 
hilight .hilight.foo 0 "'.'"            .scheme.quote 
hilight .hilight.foo 0 "'\\\\.'"        .scheme.quote ; '\?' quoted char 

hilight .hilight.foo 1 "if"             .scheme.keyword 
hilight .hilight.foo 1 "then"           .scheme.keyword 
hilight .hilight.foo 1 "else"           .scheme.keyword 
hilight .hilight.foo 1 "endif"          .scheme.keyword 

; File hook - called when new file is loaded. 
define-macro fhook-foo 
    ; Assign the hilighting 
    set-variable $buffer-hilight .hilight.foo 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 

; Notification that hook is loaded. 
ml-write "[MicroEmacs foo file hook loaded]" 

Adding a Template

A template inserts initial text into a new file that is created. This mechanism is typically used to insert a standard header into the file on creation. The insertion text is defined within a template file, given the file extension etf(8), which is created in the corresponding global, company or user directory as determined in step 1. The template is named ext.etf, so for our example file foo, the template file is called foo.etf. We shall simply add a file header, our comment is # (as defined by the hilighting tokens). Our example foo template file foo.etf may be defined as follows:-

#-!- foo -!- ################################# 
# 
#  Created By    : $USER_NAME$ 
#  Created       : $ASCII_TIME$ 
#  Last Modified : <160495.1521> 
# 
#  Description 
# 
#  Notes 
# 
#  History 
# 
#  Copyright (c) $YEAR$ $COMPANY_NAME$. 
############################################## 

The template file must be explicitly loaded by the hook file, within the fhook function. A new file condition may be tested within the fhook macro by checking the numerical argument, an argument of 0 indicates that this is a new file. The template file is inserted with an invocation of etfinsrt(3). The fhook macro checks the argument and inserts the template file as follows:-

; File hook - called when new file is loaded. 
define-macro fhook-foo 
    ; if arg is 0 this is a new file so add template 
    !if &not @# 
        etfinsrt "foo" 
    !endif 
    ; Assign the hilighting 
    set-variable $buffer-hilight .hilight.foo 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 

Adding abbreviations

Abbreviations are short-cut expansions which may be defined for the language specific file. The abbreviations are defined in a eaf(8) file, ext.eaf, located in the appropriately defined MicroEmacs directory. The abbreviation file defines the key sequences which may be automatically inserted, under user intervention, using expand-abbrev(2). An abbreviation file for foo, foo.eaf, may be defined as:-

if "if \p\rthen\rendif\P" 
el "else\r\p\P" 

The binding to the hook is defined in the fhook macro using buffer-abbrev-file(2). For the example language file foo the fhook macro becomes:-

; File hook - called when new file is loaded. 
define-macro fhook-foo 
    ; if arg is 0 this is a new file so add template 
    !if &not @# 
        etfinsrt "foo" 
    !endif 
    ; Assign the hilighting 
    set-variable $buffer-hilight .hilight.foo 
    ; Set the abbreviation file 
    buffer-abbrev-file "foo" 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 

Automatic Indentation

Automatic indentation may be applied to the file, such that the indentation is automatically performed when new lines are entered into the file. Indentation also benefits from automatic re-styling operations using restyle-region(3) and restyle-buffer(3).

The indentation style is declared by defining language tokens that constitute positions in the syntax where the indentation is changed. The indentation requires a unique identifier to identify the indentation style, the hilighting identifier is used. If hilighting is not defined, then the language template may still obtain an identifier as described in the hilighting section.

The indention is create with an argument of 0 to the indent(2) command, the subsequent tokens are defined using indent with no argument. For our simple foo syntax then the indentation might be defined as follows:-

0 indent  .hilight.foo 2 10 
indent .hilight.foo n "then" 4 
indent .hilight.foo s "else" -4 
indent .hilight.foo o "endif" -4 

This provides an indentation of the form:-

if condition 
then 
    XXXX 
else 
    if condition 
    then 
        XXXX 
    endif 
endif 

The indentation is bound to the buffer in the fhook macro by defining $buffer-indent(5). For the example file foo then the fhook is defined as:-

; File hook - called when new file is loaded. 
define-macro fhook-foo 
    ; if arg is 0 this is a new file so add template 
    !if &not @# 
        etfinsrt "foo" 
    !endif 
    ; Assign the hilighting 
    set-variable $buffer-hilight .hilight.foo 
    ; Assign the buffer indentation 
    set-variable $buffer-indent .hilight.foo 
    ; Set the abbreviation file 
    buffer-abbrev-file "foo" 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 

Setting Buffer Modes

Buffer modes which are to be adopted (or discarded) by the language specific file are defined in the fhook macro. Typical modes that are applied are:-

time


Enables time stamping on the file, modifying the time stamp field with the modification date and time.

indent


Automatic indentation, where the cursor is returned to the same column on entering a new line, rather than to the start of the line.

As an example, the foo fhook file becomes:-

; File hook - called when new file is loaded. 
define-macro fhook-foo 
    ; if arg is 0 this is a new file so add template 
    !if &not @# 
        etfinsrt "foo" 
    !endif 
    ; Assign the hilighting 
    set-variable $buffer-hilight .hilight.foo 
    ; Assign the buffer indentation 
    set-variable $buffer-indent .hilight.foo 
    ; Set the abbreviation file 
    buffer-abbrev-file "foo" 
    ; Set up the buffer modes 
    1 buffer-mode "time" 
    1 buffer-mode "indent" 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 

Assigning New Bindings

New bindings and language specific macros may be added to the language specific file. New macros, to extend the repertoire of commands specifically developed for the language file are defined within the macro body using define-macro(2) these are automatically loaded when the hook file is loaded, which in turn is loaded when the file type is identified and loaded.

New bindings, which may be associated with new macros or existing commands, are assigned within the fhook macro. As an example, we shall extend the foo language file to include a commenting and uncommenting macros, locally binding the macros to the keys "C-c  C-c" and "C-c C-d" respectively. The macro definitions are defined as follows:-

; Macro to comment a line 
define-macro foo-comment-line 
    !while &gre &pdec @# 1 0 
        beginning-of-line 
        insert-string "#" 
        beginning-of-line 
        forward-line 
    !done 
!emacro 

; Macro to remove a comment from a line 
define-macro foo-uncomment-line 
    !while &gre &pdec @# 1 0 
        beginning-of-line 
        -1 search-forward "#" 
        backward-delete-char 
        forward-line 
    !done 
!emacro 

The key bindings for the macros are defined for the local buffer ONLY, as such are added using buffer-bind-key(2). The bindings are declared in the fhook macro as follows:-

; File hook - called when new file is loaded. 
define-macro fhook-foo 
    ; if arg is 0 this is a new file so add template 
    !if &not @# 
        etfinsrt "foo" 
    !endif 
    ; Assign the hilighting 
    set-variable $buffer-hilight .hilight.foo 
    ; Assign the buffer indentation 
    set-variable $buffer-indent .hilight.foo 
    ; Set the abbreviation file 
    buffer-abbrev-file "foo" 
    ; Set up the buffer modes 
    1 buffer-mode "time" 
    1 buffer-mode "indent" 
    ; Set up local bindings 
    buffer-bind-key foo-comment-line "C-c C-c" 
    buffer-bind-key foo-uncomment-line "C-c C-d" 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 

Allowing Other to Modify the Hook

Other users of the file hook may need to modify or extend the file hook, the most common form is the addition of user specific hilight tokens. MicroEmacs uses a simple mechanism of executing a user hook extension file if it exists. The extension file name must be of the form myXXX.emf, i.e. for our example it must be "myfoo.emf". This is performed at the end of the macro file so that anything within the file can be altered, it is executed as follows:-

; load in user extensions if found 
!force execute-file "myfoo" 

Note the !force(4) directive is used as the file may not exist.

Summing Up

The previous sections have presented the basic steps involved in setting up a new language file template. They cater for simple file types, for more complex examples then browse the hkxxx.emf files.

The completed files that should have been generated by following the previous examples are now presented:-

file.foo

# This is a comment. 
if condition 
then 
    do something 
else 
    if condition 
    then 
        do something 
    endif 
endif 

hkfoo.emf

!if &sequal .hilight.foo "ERROR" 
    ; Allocate a hilighting scheme number 
    set-variable .hilight.foo &pinc .hilight.next 1 
!endif 

; Define the hilighting scheme 
0 hilight .hilight.foo 1                $global-scheme 
hilight .hilight.foo 2 "#"              .scheme.comment 
hilight .hilight.foo 4 "\"" "\"" "\\"   .scheme.string 
hilight .hilight.foo 0 "'.'"            .scheme.quote 
hilight .hilight.foo 0 "'\\\\.'"        .scheme.quote ; '\?' quoted char 

hilight .hilight.foo 1 "if"             .scheme.keyword 
hilight .hilight.foo 1 "then"           .scheme.keyword 
hilight .hilight.foo 1 "else"           .scheme.keyword 
hilight .hilight.foo 1 "endif"          .scheme.keyword 

; File hook - called when new file is loaded. 
define-macro fhook-foo 
    ; Assign the hilighting 
    set-variable $buffer-hilight .hilight.foo 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 

; Define the indentation scheme 
0 indent  .hilight.foo 2 10 
indent .hilight.foo n "then" 4 
indent .hilight.foo s "else" -4 
indent .hilight.foo o "endif" -4 

; Reset the hilighting printer format and define the color bindings. 
0 hilight-print .hilight.foo 
hilight-print .hilight.foo "i"  .scheme.comment 
hilight-print .hilight.foo "b"  .scheme.keyword 
hilight-print .hilight.foo "bi" .scheme.string .scheme.quote 

; Macro to comment a line 
define-macro foo-comment-line 
    !while &gre &pdec @# 1 0 
        beginning-of-line 
        insert-string "#" 
        beginning-of-line 
        forward-line 
    !done 
!emacro 

; Macro to remove a comment from a line 
define-macro foo-uncomment-line 
    !while &gre &pdec @# 1 0 
        beginning-of-line 
        -1 search-forward "#" 
        backward-delete-char 
        forward-line 
    !done 
!emacro 

; File hook - called when new file is loaded. 
define-macro fhook-foo 
    ; if arg is 0 this is a new file so add template 
    !if &not @# 
        etfinsrt "foo" 
    !endif 
    ; Assign the hilighting 
    set-variable $buffer-hilight .hilight.foo 
    ; Assign the buffer indentation 
    set-variable $buffer-indent .hilight.foo 
    ; Set the abbreviation file 
    buffer-abbrev-file "foo" 
    ; Set up the buffer modes 
    1 buffer-mode "time" 
    1 buffer-mode "indent" 
    ; Set up local bindings 
    buffer-bind-key foo-comment-line "C-c C-c" 
    buffer-bind-key foo-uncomment-line "C-c C-d" 
    ; Temporary comment to make sure that it works. 
    ml-write "Loaded a foo file" 
!emacro 

; Notification that hook is loaded. 
ml-write "[MicroEmacs foo file hook loaded]" 

; load in user extensions if found 
!force execute-file "myfoo" 

foo.eaf

if "if \p\rthen\rendif\P" 
el "else\r\p\P" 

foo.etf

#-!- foo -!- ################################# 
# 
#  Created By    : $USER_NAME$ 
#  Created       : $ASCII_TIME$ 
#  Last Modified : <160495.1521> 
# 
#  Description 
# 
#  Notes 
# 
#  History 
# 
#  Copyright (c) $YEAR$ $COMPANY_NAME$. 
############################################## 

SEE ALSO

add-file-hook(2), buffer-abbrev-file(2), buffer-init(3), etfinsrt(3), execute-buffer(2), expand-abbrev(2), global-abbrev-file(2), hilight(2), scheme-editor(3), indent(2), indent(2m), restyle-buffer(3), restyle-region(3), time(2m), $buffer-hilight(5), $buffer-indent(5), etf(8), eaf(8),
File Hooks.

(c) Copyright JASSPA 2009
Last Modified: 2009/08/29
Generated On: 2009/10/12