Linux.pl
Opcje wyszukiwania podręcznika man:
Lista stron man zaczynających się od znaku:
A   B   C   D   E   F   G   H   I   J   K   L   M   N   O   P   Q   R   S   T   U   V   W   X   Y   Z   ALPHA   NUM   OTHER   ALL
LEX(P)                     POSIX Programmer's Manual                    LEX(P)

NAME
       lex - generate programs for lexical tasks (DEVELOPMENT)

SYNOPSIS
       lex [-t][-n|-v][file ...]

DESCRIPTION
       The  lex  utility  shall generate C programs to be used in lexical pro-
       cessing of character input, and that can be used  as  an  interface  to
       yacc.  The  C programs shall be generated from lex source code and con-
       form to the ISO C standard. Usually, the lex utility  shall  write  the
       program  it  generates  to the file lex.yy.c; the state of this file is
       unspecified if lex exits with a non-zero exit status. See the  EXTENDED
       DESCRIPTION  section  for  a complete description of the lex input lan-
       guage.

OPTIONS
       The lex utility  shall  conform  to  the  Base  Definitions  volume  of
       IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines.

       The following options shall be supported:

       -n     Suppress  the  summary of statistics usually written with the -v
              option. If no table sizes are specified in the lex  source  code
              and the -v option is not specified, then -n is implied.

       -t     Write  the  resulting  program  to  standard  output  instead of
              lex.yy.c.

       -v     Write a summary of lex statistics to the standard  output.  (See
              the  discussion  of  lex table sizes in Definitions in lex .) If
              the -t option is specified and -n is not specified, this  report
              shall be written to standard error. If table sizes are specified
              in the lex source code, and if the -n option is  not  specified,
              the -v option may be enabled.

OPERANDS
       The following operand shall be supported:

       file   A pathname of an input file. If more than one such file is spec-
              ified, all files shall be concatenated to produce a  single  lex
              program. If no file operands are specified, or if a file operand
              is '-' , the standard input shall be used.

STDIN
       The standard input shall be used if no file operands are specified,  or
       if a file operand is '-' . See INPUT FILES.

INPUT FILES
       The  input  files  shall  be  text files containing lex source code, as
       described in the EXTENDED DESCRIPTION section.

ENVIRONMENT VARIABLES
       The following environment variables shall affect the execution of lex:

       LANG   Provide a default value for the  internationalization  variables
              that  are  unset  or  null.  (See the Base Definitions volume of
              IEEE Std 1003.1-2001, Section  8.2,  Internationalization  Vari-
              ables  for the precedence of internationalization variables used
              to determine the values of locale categories.)

       LC_ALL If set to a non-empty string value, override the values  of  all
              the other internationalization variables.

       LC_COLLATE

              Determine  the  locale  for  the behavior of ranges, equivalence
              classes, and multi-character collating elements  within  regular
              expressions.  If  this  variable is not set to the POSIX locale,
              the results are unspecified.

       LC_CTYPE
              Determine the locale for  the  interpretation  of  sequences  of
              bytes  of  text  data as characters (for example, single-byte as
              opposed to multi-byte characters in arguments and input  files),
              and  the  behavior  of  character classes within regular expres-
              sions.  If this variable is not set to  the  POSIX  locale,  the
              results are unspecified.

       LC_MESSAGES
              Determine  the  locale  that should be used to affect the format
              and contents of diagnostic messages written to standard error.

       NLSPATH
              Determine the location of message catalogs for the processing of
              LC_MESSAGES .

ASYNCHRONOUS EVENTS
       Default.

STDOUT
       If the -t option is specified, the text file of C source code output of
       lex shall be written to standard output.

       If the -t option is not specified:

        * Implementation-defined informational, error,  and  warning  messages
          concerning the contents of lex source code input shall be written to
          either the standard output or standard error.

        * If the -v option is specified and the -n option  is  not  specified,
          lex  statistics  shall also be written to either the standard output
          or  standard  error,  in  an  implementation-defined  format.  These
          statistics may also be generated if table sizes are specified with a
          '%' operator in the Definitions section, as long as the -n option is
          not specified.

STDERR
       If  the  -t  option is specified, implementation-defined informational,
       error, and warning messages concerning the contents of lex source  code
       input shall be written to the standard error.

       If the -t option is not specified:

        1. Implementation-defined  informational,  error, and warning messages
           concerning the contents of lex source code input shall  be  written
           to either the standard output or standard error.

        2. If  the  -v option is specified and the -n option is not specified,
           lex statistics shall also be written to either the standard  output
           or  standard  error,  in  an  implementation-defined  format. These
           statistics may also be generated if table sizes are specified  with
           a '%' operator in the Definitions section, as long as the -n option
           is not specified.

OUTPUT FILES
       A text file containing C source code shall be written to  lex.yy.c,  or
       to the standard output if the -t option is present.

EXTENDED DESCRIPTION
       Each input file shall contain lex source code, which is a table of reg-
       ular expressions with corresponding actions in the form  of  C  program
       fragments.

       When  lex.yy.c  is  compiled and linked with the lex library (using the
       -l l operand with c99), the  resulting  program  shall  read  character
       input  from the standard input and shall partition it into strings that
       match the given expressions.

       When an expression is matched, these actions shall occur:

        * The input string that was matched shall be left in yytext as a null-
          terminated  string;  yytext  shall  either  be an external character
          array or a pointer to a character string. As  explained  in  Defini-
          tions  in lex , the type can be explicitly selected using the %array
          or %pointer declarations, but the default is implementation-defined.

        * The  external  int yyleng shall be set to the length of the matching
          string.

        * The expression's corresponding program fragment, or action, shall be
          executed.

       During  pattern  matching, lex shall search the set of patterns for the
       single longest possible match. Among rules that match the  same  number
       of characters, the rule given first shall be chosen.

       The general format of lex source shall be:

              Definitions
              %%
              Rules
              %%
              UserSubroutines

       The  first "%%" is required to mark the beginning of the rules (regular
       expressions and actions); the second "%%" is required only if user sub-
       routines follow.

       Any  line  in the Definitions section beginning with a <blank> shall be
       assumed to be a C program fragment and shall be copied to the  external
       definition area of the lex.yy.c file.  Similarly, anything in the Defi-
       nitions section included between delimiter lines containing  only  "%{"
       and "%}" shall also be copied unchanged to the external definition area
       of the lex.yy.c file.

       Any such input (beginning with a <blank> or within "%{" and "%}" delim-
       iter  lines) appearing at the beginning of the Rules section before any
       rules are specified shall be written to lex.yy.c after the declarations
       of variables for the yylex() function and before the first line of code
       in yylex(). Thus, user variables local to yylex() can be declared here,
       as well as application code to execute upon entry to yylex().

       The  action  taken  by lex when encountering any input beginning with a
       <blank> or within "%{" and "%}" delimiter lines appearing in the  Rules
       section  but  coming after one or more rules is undefined. The presence
       of such input may result in an  erroneous  definition  of  the  yylex()
       function.

   Definitions in lex
       Definitions  appear  before  the first "%%" delimiter. Any line in this
       section not contained between "%{" and "%}"  lines  and  not  beginning
       with  a  <blank>  shall be assumed to define a lex substitution string.
       The format of these lines shall be:

              name substitute

       If a name does not meet the requirements for identifiers in  the  ISO C
       standard,  the result is undefined. The string substitute shall replace
       the string { name} when it is used in a rule. The name string shall  be
       recognized  in  this context only when the braces are provided and when
       it does not appear within a bracket expression or within double-quotes.

       In  the  Definitions  section,  any  line beginning with a '%' (percent
       sign) character and followed by an  alphanumeric  word  beginning  with
       either  's'  or  'S'  shall  define a set of start conditions. Any line
       beginning with a '%' followed by a word beginning with  either  'x'  or
       'X'  shall  define a set of exclusive start conditions. When the gener-
       ated scanner is in a %s state, patterns with no state  specified  shall
       be  also  active; in a %x state, such patterns shall not be active. The
       rest of the line, after the first word, shall be considered to  be  one
       or  more  <blank>-separated  names of start conditions. Start condition
       names shall be constructed in the same way as definition  names.  Start
       conditions  can be used to restrict the matching of regular expressions
       to one or more states as described in Regular Expressions in lex .

       Implementations shall accept either  of  the  following  two  mutually-
       exclusive declarations in the Definitions section:

       %array Declare  the  type  of  yytext to be a null-terminated character
              array.

       %pointer
              Declare the type of yytext to be a pointer to a  null-terminated
              character string.

       The default type of yytext is implementation-defined. If an application
       refers to yytext outside of the scanner source file (that  is,  via  an
       extern),  the  application  shall  include  the  appropriate  %array or
       %pointer declaration in the scanner source file.

       Implementations shall accept declarations in  the  Definitions  section
       for setting certain internal table sizes. The declarations are shown in
       the following table.

                        Table: Table Size Declarations in lex

           Declaration  Description                         Minimum Value
           %p n         Number of positions                 2500
           %n n         Number of states                    500
           %a n         Number of transitions               2000
           %e n         Number of parse tree nodes          1000
           %k n         Number of packed character classes  1000
           %o n         Size of the output array            3000

       In the table, n represents a positive decimal integer, preceded by  one
       or  more  <blank>s.  The  exact  meaning of these table size numbers is
       implementation-defined. The implementation  shall  document  how  these
       numbers  affect  the lex utility and how they are related to any output
       that may be generated  by  the  implementation  should  limitations  be
       encountered during the execution of lex. It shall be possible to deter-
       mine from this output which of the table size values needs to be  modi-
       fied  to  permit lex to successfully generate tables for the input lan-
       guage.  The values in the column Minimum  Value  represent  the  lowest
       values conforming implementations shall provide.

   Rules in lex
       The rules in lex source files are a table in which the left column con-
       tains regular expressions and the right column contains actions (C pro-
       gram fragments) to be executed when the expressions are recognized.

              ERE action
              ERE action...

       The  extended  regular expression (ERE) portion of a row shall be sepa-
       rated from action by one or more <blank>s. A  regular  expression  con-
       taining  <blank>s shall be recognized under one of the following condi-
       tions:

        * The entire expression appears within double-quotes.

        * The <blank>s appear within double-quotes or square brackets.

        * Each <blank> is preceded by a backslash character.

   User Subroutines in lex
       Anything in the user subroutines section shall be  copied  to  lex.yy.c
       following yylex().

   Regular Expressions in lex
       The  lex  utility shall support the set of extended regular expressions
       (see the Base Definitions volume of IEEE Std 1003.1-2001, Section  9.4,
       Extended  Regular Expressions), with the following additions and excep-
       tions to the syntax:

       "..."  Any string enclosed in double-quotes shall represent the charac-
              ters  within  the double-quotes as themselves, except that back-
              slash escapes (which appear in the  following  table)  shall  be
              recognized.   Any  backslash-escape sequence shall be terminated
              by the closing quote. For example, "\01" "1" represents a single
              string: the octal value 1 followed by the character '1' .

       <state>r, <state1,state2,...>r

              The  regular expression r shall be matched only when the program
              is in one of the start conditions indicated  by  state,  state1,
              and  so  on;  see Actions in lex . (As an exception to the typo-
              graphical  conventions  of  the   rest   of   this   volume   of
              IEEE Std 1003.1-2001,  in this case <state> does not represent a
              metavariable, but the literal angle-bracket characters surround-
              ing  a  symbol.) The start condition shall be recognized as such
              only at the beginning of a regular expression.

       r/x    The regular expression r shall be matched only if it is followed
              by  an occurrence of regular expression x ( x is the instance of
              trailing context, further defined below).  The token returned in
              yytext  shall only match r. If the trailing portion of r matches
              the beginning of x, the result is unspecified. The r  expression
              cannot  include  further trailing context or the '$' (match-end-
              of-line) operator; x cannot include the '^' (match-beginning-of-
              line) operator, nor trailing context, nor the '$' operator. That
              is, only one occurrence of trailing context is allowed in a  lex
              regular expression, and the '^' operator only can be used at the
              beginning of such an expression.

       {name} When name is one of the substitution symbols  from  the  Defini-
              tions section, the string, including the enclosing braces, shall
              be replaced by the substitute value. The substitute value  shall
              be  treated  in  the  extended  regular expression as if it were
              enclosed in parentheses. No substitution shall occur if {  name}
              occurs within a bracket expression or within double-quotes.

       Within  an  ERE,  a backslash character shall be considered to begin an
       escape sequence as specified in the table in the Base Definitions  vol-
       ume  of  IEEE Std 1003.1-2001, Chapter 5, File Format Notation ( '\\' ,
       '\a' , '\b' , '\f' , '\n' , '\r' , '\t' ,  '\v'  ).  In  addition,  the
       escape sequences in the following table shall be recognized.

       A  literal  <newline>  cannot  occur within an ERE; the escape sequence
       '\n' can be used to represent a <newline>. A  <newline>  shall  not  be
       matched by a period operator.

                           Table: Escape Sequences in lex

       Escape
       Sequence Description                    Meaning
       \digits  A backslash character followed The character whose encoding
                by the longest sequence of     is represented by the one,
                one, two, or three octal-digit two, or three-digit octal
                characters (01234567). If all  integer. If the size of a byte
                of the digits are 0 (that is,  on the system is greater than
                representation of the NUL      nine bits, the valid escape
                character), the behavior is    sequence used to represent a
                undefined.                     byte is implementation-
                                               defined. Multi-byte characters
                                               require multiple, concatenated
                                               escape sequences of this type,
                                               including the leading '\' for
                                               each byte.
       \xdigits A backslash character followed The character whose encoding
                by the longest sequence of     is represented by the hexadec-
                hexadecimal-digit characters   imal integer.
                (01234567abcdefABCDEF). If all
                of the digits are 0 (that is,
                representation of the NUL
                character), the behavior is
                undefined.
       \c       A backslash character followed The character 'c' , unchanged.
                by any character not described
                in this table or in the table
                in the Base Definitions volume
                of IEEE Std 1003.1-2001, Chap-
                ter 5, File Format Notation (
                '\\' , '\a' , '\b' , '\f' ,
                '\n' , '\r' , '\t' , '\v' ).

       Note:  If  a  '\x'  sequence needs to be immediately followed by a hex-
              adecimal digit character, a sequence such as "\x1"  "1"  can  be
              used,  which represents a character containing the value 1, fol-
              lowed by the character '1' .

       The order of precedence given to extended regular expressions  for  lex
       differs   from  that  specified  in  the  Base  Definitions  volume  of
       IEEE Std 1003.1-2001, Section 9.4, Extended  Regular  Expressions.  The
       order  of  precedence for lex shall be as shown in the following table,
       from high to low.

       Note:  The escaped characters entry is not meant to  imply  that  these
              are  operators, but they are included in the table to show their
              relationships to the true operators. The start condition, trail-
              ing  context, and anchoring notations have been omitted from the
              table because of the placement restrictions  described  in  this
              section;  they  can only appear at the beginning or ending of an
              ERE.

                                Table: ERE Precedence in lex

                  Extended Regular Expression        Precedence
                  collation-related bracket symbols  [= =] [: :] [. .]
                  escaped characters                 \<special character>
                  bracket expression                 [ ]
                  quoting                            "..."
                  grouping                           ( )
                  definition                         {name}
                  single-character RE duplication    * + ?
                  concatenation
                  interval expression                {m,n}
                  alternation                        |

       The ERE anchoring operators '^' and '$' do not  appear  in  the  table.
       With  lex  regular expressions, these operators are restricted in their
       use: the '^' operator can only be used at the beginning  of  an  entire
       regular expression, and the '$' operator only at the end. The operators
       apply to the entire regular expression. Thus, for example, the  pattern
       "(^abc)|(def$)" is undefined; it can instead be written as two separate
       rules, one with the regular expression "^abc" and  one  with  "def$"  ,
       which  share a common action via the special '|' action (see below). If
       the pattern were written "^abc|def$" , it would match either  "abc"  or
       "def" on a line by itself.

       Unlike the general ERE rules, embedded anchoring is not allowed by most
       historical lex implementations. An example of embedded anchoring  would
       be  for  patterns such as "(^| )foo( |$)" to match "foo" when it exists
       as a complete word. This functionality can be obtained  using  existing
       lex features:

              ^foo/[ \n]      |
              " foo"/[ \n]    /* Found foo as a separate word. */

       Note  also  that '$' is a form of trailing context (it is equivalent to
       "/\n" ) and as such cannot be used with regular expressions  containing
       another  instance  of  the  operator  (see  the preceding discussion of
       trailing context).

       The additional regular expressions trailing-context operator '/' can be
       used  as an ordinary character if presented within double-quotes, "/" ;
       preceded by a backslash, "\/" ; or within a bracket expression, "[/]" .
       The  start-condition  '<'  and '>' operators shall be special only in a
       start condition at the beginning of a regular expression; elsewhere  in
       the regular expression they shall be treated as ordinary characters.

   Actions in lex
       The  action to be taken when an ERE is matched can be a C program frag-
       ment or the special actions described below; the program  fragment  can
       contain one or more C statements, and can also include special actions.
       The empty C statement ';' shall be a valid action; any  string  in  the
       lex.yy.c  input  that  matches  the  pattern  portion of such a rule is
       effectively ignored or skipped. However, the absence of an action shall
       not  be  valid,  and  the action lex takes in such a condition is unde-
       fined.

       The specification for an action, including  C  statements  and  special
       actions, can extend across several lines if enclosed in braces:

              ERE <one or more blanks> { program statement
                                         program statement }

       The  default action when a string in the input to a lex.yy.c program is
       not matched by any expression shall be to copy the string to  the  out-
       put.  Because  the default behavior of a program generated by lex is to
       read the input and copy it to the output, a minimal lex source  program
       that  has  just  "%%" shall generate a C program that simply copies the
       input to the output unchanged.

       Four special actions shall be available:

              |   ECHO;   REJECT;   BEGIN

       |      The action '|' means that the action for the next  rule  is  the
              action for this rule. Unlike the other three actions, '|' cannot
              be enclosed in braces or be semicolon-terminated;  the  applica-
              tion  shall  ensure  that  it  is specified alone, with no other
              actions.

       ECHO;  Write the contents of the string yytext on the output.

       REJECT;
              Usually only a single expression is matched by a given string in
              the  input.  REJECT  means "continue to the next expression that
              matches the current input", and shall cause  whatever  rule  was
              the  second choice after the current rule to be executed for the
              same input. Thus, multiple rules can be matched and executed for
              one  input  string  or  overlapping input strings.  For example,
              given the regular expressions "xyz" and "xy" and the input "xyz"
              ,  usually  only  the  regular expression "xyz" would match. The
              next attempted match would start after z. If the last action  in
              the "xyz" rule is REJECT, both this rule and the "xy" rule would
              be executed. The REJECT action may  be  implemented  in  such  a
              fashion  that  flow of control does not continue after it, as if
              it were equivalent to a goto to another part of yylex(). The use
              of REJECT may result in somewhat larger and slower scanners.

       BEGIN  The action:

              BEGIN newstate;

       switches  the  state  (start condition) to newstate. If the string new-
       state has not been declared previously as a start condition in the Def-
       initions  section,  the  results  are unspecified. The initial state is
       indicated by the digit '0' or the token INITIAL.

       The functions or macros described below are  accessible  to  user  code
       included in the lex input. It is unspecified whether they appear in the
       C code output of lex, or are accessible only through the  -l l  operand
       to c99 (the lex library).

       int  yylex(void)

              Performs  lexical  analysis  on  the  input; this is the primary
              function generated by the lex utility. The function shall return
              zero  when  the  end  of  input  is reached; otherwise, it shall
              return non-zero values (tokens) determined by the  actions  that
              are selected.

       int  yymore(void)

              When called, indicates that when the next input string is recog-
              nized, it is to be appended  to  the  current  value  of  yytext
              rather  than replacing it; the value in yyleng shall be adjusted
              accordingly.

       int  yyless(int  n)

              Retains n initial  characters  in  yytext,  NUL-terminated,  and
              treats  the  remaining  characters as if they had not been read;
              the value in yyleng shall be adjusted accordingly.

       int  input(void)

              Returns the next character from the input, or  zero  on  end-of-
              file.   It  shall  obtain  input  from  the stream pointer yyin,
              although possibly via an intermediate buffer. Thus,  once  scan-
              ning  has  begun,  the  effect  of altering the value of yyin is
              undefined. The character read shall be removed  from  the  input
              stream of the scanner without any processing by the scanner.

       int  unput(int  c)

              Returns  the  character  'c' to the input; yytext and yyleng are
              undefined until the next expression is matched.  The  result  of
              using  unput()  for  more  characters  than  have  been input is
              unspecified.

       The following functions shall appear only in the lex library accessible
       through the -l l operand; they can therefore be redefined by a conform-
       ing application:

       int  yywrap(void)

              Called by yylex() at end-of-file;  the  default  yywrap()  shall
              always return 1. If the application requires yylex() to continue
              processing with another source of input,  then  the  application
              can  include  a function yywrap(), which associates another file
              with the external variable FILE * yyin and shall return a  value
              of zero.

       int  main(int  argc, char *argv[])

              Calls  yylex() to perform lexical analysis, then exits. The user
              code can contain main() to perform  application-specific  opera-
              tions, calling yylex() as applicable.

       Except  for input(), unput(), and main(), all external and static names
       generated by lex shall begin with the prefix yy or YY.

EXIT STATUS
       The following exit values shall be returned:

        0     Successful completion.

       >0     An error occurred.

CONSEQUENCES OF ERRORS
       Default.

       The following sections are informative.

APPLICATION USAGE
       Conforming applications are warned that in the Rules  section,  an  ERE
       without  an action is not acceptable, but need not be detected as erro-
       neous by lex. This may result in compilation or runtime errors.

       The purpose of input() is to take characters off the input  stream  and
       discard  them as far as the lexical analysis is concerned. A common use
       is to discard the body of a comment once the beginning of a comment  is
       recognized.

       The lex utility is not fully internationalized in its treatment of reg-
       ular expressions in the lex source code or generated lexical  analyzer.
       It would seem desirable to have the lexical analyzer interpret the reg-
       ular expressions given in the lex source according to  the  environment
       specified when the lexical analyzer is executed, but this is not possi-
       ble with the current lex technology. Furthermore, the  very  nature  of
       the lexical analyzers produced by lex must be closely tied to the lexi-
       cal requirements of the input language being described, which  is  fre-
       quently  locale-specific anyway. (For example, writing an analyzer that
       is used for French text is  not  automatically  useful  for  processing
       other languages.)

EXAMPLES
       The following is an example of a lex program that implements a rudimen-
       tary scanner for a Pascal-like syntax:

              %{
              /* Need this for the call to atof() below. */
              #include <math.h>
              /* Need this for printf(), fopen(), and stdin below. */
              #include <stdio.h>
              %}

              DIGIT    [0-9]
              ID       [a-z][a-z0-9]*

              %%

              {DIGIT}+ {
                  printf("An integer: %s (%d)\n", yytext,
                      atoi(yytext));
                  }

              {DIGIT}+"."{DIGIT}*        {
                  printf("A float: %s (%g)\n", yytext,
                      atof(yytext));
                  }

              if|then|begin|end|procedure|function        {
                  printf("A keyword: %s\n", yytext);
                  }

              {ID}    printf("An identifier: %s\n", yytext);

              "+"|"-"|"*"|"/"        printf("An operator: %s\n", yytext);

              "{"[^}\n]*"}"    /* Eat up one-line comments. */

              [ \t\n]+        /* Eat up white space. */

              .  printf("Unrecognized character: %s\n", yytext);

              %%

              int main(int argc, char *argv[])
              {
                  ++argv, --argc;  /* Skip over program name. */
                  if (argc > 0)
                      yyin = fopen(argv[0], "r");
                  else
                      yyin = stdin;

                  yylex();
              }

RATIONALE
       Even though the -c option and references to the C language are retained
       in  this description, lex may be generalized to other languages, as was
       done at one time for EFL, the Extended FORTRAN Language. Since the  lex
       input  specification  is  essentially language-independent, versions of
       this utility could be written to produce Ada, Modula-2, or Pascal code,
       and there are known historical implementations that do so.

       The  current  description  of  lex  bypasses  the issue of dealing with
       internationalized EREs in the lex source code or generated lexical ana-
       lyzer.  If it follows the model used by awk (the source code is assumed
       to be presented in the POSIX locale, but input and output  are  in  the
       locale  specified by the environment variables), then the tables in the
       lexical analyzer produced by lex would interpret EREs specified in  the
       lex source in terms of the environment variables specified when lex was
       executed. The desired effect would be  to  have  the  lexical  analyzer
       interpret the EREs given in the lex source according to the environment
       specified when the lexical analyzer is executed, but this is not possi-
       ble with the current lex technology.

       The  description of octal and hexadecimal-digit escape sequences agrees
       with the ISO C standard use of escape sequences. See the RATIONALE  for
       ed  for  a  discussion of bytes larger than 9 bits being represented by
       octal values.  Hexadecimal values can represent larger bytes and multi-
       byte characters directly, using as many digits as required.

       There is no detailed output format specification. The observed behavior
       of lex under four different historical implementations was that none of
       these  implementations consistently reported the line numbers for error
       and warning messages.  Furthermore, there was  a  desire  that  lex  be
       allowed  to output additional diagnostic messages. Leaving message for-
       mats unspecified avoids these formatting questions  and  problems  with
       internationalization.

       Although the %x specifier for exclusive start conditions is not histor-
       ical practice, it is believed to be a minor change to historical imple-
       mentations  and greatly enhances the usability of lex programs since it
       permits an application to obtain the expected functionality with  fewer
       statements.

       The %array and %pointer declarations were added as a compromise between
       historical systems. The System V-based lex copies the matched text to a
       yytext  array. The flex program, supported in BSD and GNU systems, uses
       a pointer. In the latter case, significant performance improvements are
       available for some scanners. Most historical programs should require no
       change in porting from one system to another because the  string  being
       referenced  is  null-terminated in both cases. (The method used by flex
       in its case is to null-terminate the token in place by remembering  the
       character  that  used  to  come  right after the token and replacing it
       before continuing on to the next scan.) Multi-file programs with exter-
       nal  references  to  yytext outside the scanner source file should con-
       tinue to operate on their historical systems, but would require one  of
       the new declarations to be considered strictly portable.

       The  description  of EREs avoids unnecessary duplication of ERE details
       because their meanings within a lex ERE are the same as  that  for  the
       ERE in this volume of IEEE Std 1003.1-2001.

       The  reason  for the undefined condition associated with text beginning
       with a <blank> or within "%{" and "%}" delimiter lines appearing in the
       Rules  section  is  historical  practice. Both the BSD and System V lex
       copy the indented (or enclosed) input in the Rules section  (except  at
       the  beginning)  to unreachable areas of the yylex() function (the code
       is written directly after a break statement). In some cases, the System
       V  lex  generates  an error message or a syntax error, depending on the
       form of indented input.

       The intention in breaking the list of functions  into  those  that  may
       appear in lex.yy.c versus those that only appear in libl.a is that only
       those functions in libl.a can be reliably  redefined  by  a  conforming
       application.

       The  descriptions  of  standard  output and standard error are somewhat
       complicated because historical lex implementations chose to issue diag-
       nostic   messages   to   standard   output   (unless   -t  was  given).
       IEEE Std 1003.1-2001 allows this behavior, but leaves  an  opening  for
       the  more  expected  behavior  of using standard error for diagnostics.
       Also, the System V behavior of writing the statistics  when  any  table
       sizes are given is allowed, while BSD-derived systems can avoid it. The
       programmer can always precisely obtain the  desired  results  by  using
       either the -t or -n options.

       The  OPERANDS  section  does  not mention the use of - as a synonym for
       standard input; not all historical implementations support  such  usage
       for any of the file operands.

       A description of the translation table was deleted from early proposals
       because of its relatively low usage in historical applications.

       The change to the  definition  of  the  input()  function  that  allows
       buffering of input presents the opportunity for major performance gains
       in some applications.

       The following examples clarify  the  differences  between  lex  regular
       expressions  and regular expressions appearing elsewhere in this volume
       of IEEE Std 1003.1-2001. For regular expressions of the  form  "r/x"  ,
       the  string matching r is always returned; confusion may arise when the
       beginning of x matches the trailing portion of r.  For  example,  given
       the  regular  expression "a*b/cc" and the input "aaabcc" , yytext would
       contain the string "aaab" on this match. But given the regular  expres-
       sion  "x*/xy" and the input "xxxy" , the token xxx, not xx, is returned
       by some implementations because xxx matches "x*" .

       In the rule "ab*/bc" , the "b*" at the end of r extends r's match  into
       the beginning of the trailing context, so the result is unspecified. If
       this rule were "ab/bc" , however, the rule matches the text  "ab"  when
       it  is followed by the text "bc" . In this latter case, the matching of
       r cannot extend into the beginning of x, so the result is specified.

FUTURE DIRECTIONS
       None.

SEE ALSO
       c99 , ed , yacc

COPYRIGHT
       Portions of this text are reprinted and reproduced in  electronic  form
       from IEEE Std 1003.1, 2003 Edition, Standard for Information Technology
       -- Portable Operating System Interface (POSIX),  The  Open  Group  Base
       Specifications  Issue  6,  Copyright  (C) 2001-2003 by the Institute of
       Electrical and Electronics Engineers, Inc and The Open  Group.  In  the
       event of any discrepancy between this version and the original IEEE and
       The Open Group Standard, the original IEEE and The Open Group  Standard
       is  the  referee document. The original Standard can be obtained online
       at http://www.opengroup.org/unix/online.html .

IEEE/The Open Group                  2003                               LEX(P)

Time taken: 0.00047 seconds


Created with the man page lookup class by Andrew Collington, php@amnuts.com