| |
Mini SQL 2.0 (Beta)
Language Specifications
Introduction
The mSQL language offers a significant subset of the
features provided by ANSI SQL. It allows a program or user to store,
manipulate and retrieve data in table structures. It does not support
some relational capabilities such as views and nested queries. Although
it does not support all the relational operations defined in the ANSI
specification, it does provide the capability of "joins" between
multiple tables.
The definitions and examples below depict mSQL key words
in upper case, but no such restriction is placed on the actual queries.
The Create Clause
The create clause as supported by mSQL 2 can be used
to create tables, indices, and sequences. It cannot be used to create
other definitions such as views. The three valid constructs of the create
clause are shown below:
- CREATE TABLE table_name (
- col_name col_type [ not null ]
- [ , col_name col_type [ not null ] ]**
- )
- CREATE [ UNIQUE ] INDEX index_name ON table_name
(
- field_name
- [ , field_name ] **
- )
- CREATE SEQUENCE ON table_name [ STEP step_val ]
[ VALUE initial_val ]
An example of the creation of a table is show below:
- CREATE TABLE emp_details (
- first_name char(15) not null,
- last_name char(15) not null,
- comment text(50),
- dept char(20),
- emp_id int
- )
The available types are:-
| char (len) |
String of characters (or other 8 bit data) |
| text (len) |
Variable length string of chracters (or other
8 bit data) The defined length is used to indicate the expected
average length of the data. Any data longer than the specified
length will be split between the data table and external overflow
buffers.
Note : text fields are slower to access
than char fields and cannot be used in an index nor in LIKE
tests.
|
| int |
Signed integer values |
| real |
Decimal or Scientific Notation real values |
The table structure shown in the example would benefit greatly from
the creation of some indices. It is assumed that the emp_id
field would be a unique value that is used to identify an employee.
Such a field would normally be defined as the primary key. mSQL 2.0
has removed support for the primary key construct within the table creation
syntax although the same result can be achieved with an index. Similarly,
a common query may be to access an employee based on the combination
of the first and last names. A compound index (i.e. constructed from
more than 1 field) would improve performance. We could construct these
indices using :
CREATE UNIQUE INDEX idx1 ON emp_details (emp_id)
CREATE INDEX idx2 ON emp_details (first_name, last_name)
These indices will be used automatically whenever a query is sent
to the database engine that uses those fields in its WHERE clause.
The user is not required to specify any special values in the query
to ensure the indices are used to increase performance.
Sequences provide a mechanism via which a sequence value can
be maintained by the mSQL server. This allows for atomic operations
(such as getting the next sequence value) and removes the concerns
associated with performing these operations in client applications.
A sequence is associated with a table and a table may contain at most
one sequence.
Once a sequence has been created it can be accessed
by SELECTing the _seq system variable from the table in which the
sequence is defined. For example
CREATE SEQUENCE ON test STEP 1 VALUE 5
SELECT _seq FROM test
The above CREATE operation would define a sequence
on the table called test that had an initial value of 5 and
would be incremented each time it is accessed (i.e. have a step of
1). The SELECT statement above would return the value 5. If the SELECT
was issued again, a value of 6 would be returned. Each time the _seq
field is selected from test the current value is returned to
the caller and the sequence value itself is incremented.
Using the STEP and VALUE options a sequence can be
created that starts at any specified number and is incremented or
decremented by any specified value. The value of a sequence would
decrease by 5 each time it was accessed if it was defined with a step
of -5.
The Drop Clause
The Drop clause is used to remove a definition from
the database. It is most commonly used to remove a table from a database
but can also be used for removing several other constructs. In 2.0 it
can be used to remove the definition of an index, a sequence, or a table.
It should be noted that dropping a table or an index removes
the data associated with that object as well as the definition.
The syntax of the drop clause as well as examples of
its use are given below.
DROP TABLE table_name
DROP INDEX index_name FROM table_name
DROP SEQUENCE FROM table_name
for example
DROP TABLE emp_details
DROP INDEX idx1 FROM emp_details
DROP SEQUENCE FROM emp_details
The Insert Clause
Unlike ANSI SQL, you cannot nest a select within an
insert (i.e. you cannot insert the data returned by a select). If you
do not specify the field names they will be used in the order they were
defined - you must specify a value for every field if you do this.
- INSERT INTO table_name [ ( column [ , column
]** ) ]
- VALUES (value [, value]** )
for example
- INSERT INTO emp_details
- (first_name, last_name, dept, salary)
- VALUES (`David', `Hughes', `Development','12345')
- INSERT INTO emp_details
- VALUES (`David', `Hughes', `Development','12345')
The number of values supplied must match the number
of columns.
The Select Clause
The SELECT offered by mSQL lacks some of the features
provided by the standard SQL specification. Development of mSQL 2 is
continuing and some of this missing functionality will be made available
in the next beta release. At this point in time, mSQL's select does
not provide
- Nested selects
- Implicit functions (e.g. count(), avg() )
It does however support:
- Joins - including table aliases
- DISTINCT row selection
- ORDER BY clauses
- Regular expression matching
- Column to Column comparisons in WHERE clauses
- Complex conditions
The formal definition of the syntax for mSQL's select
clause is
- SELECT [table.]column [ , [table.]column ]**
- FROM table [ = alias] [ , table [ = alias] ]**
- [ WHERE [table.] column OPERATOR VALUE
- [ AND | OR [table.]column OPERATOR VALUE]** ]
- [ ORDER BY [table.]column [DESC] [, [table.]column
[DESC] ]
OPERATOR can be <,> , =, <=, =, <>, LIKE,
RLIKE or CLIKE
VALUE can be a literal value or a column name
Where clauses may contain '(' ')' to nest conditions
e.g. "where (age <20 or age>30) and sex = 'male'" .
A simple select may be
- SELECT first_name, last_name FROM emp_details
- WHERE dept = `finance'
To sort the returned data in ascending order by last_name
and descending order by first_name the query would look like this
- SELECT first_name, last_name FROM emp_details
- WHERE dept = `finance'
- ORDER BY last_name, first_name DESC
And to remove any duplicate rows from the result of
the select, the DISTINCT operator could be used:
- SELECT DISTINCT first_name, last_name FROM emp_details
- WHERE dept = `finance'
- ORDER BY last_name, first_name DESC
mSQL provides three regular expression operators for
use in where comparisons. The standard SQL syntax provides
a very simplistic regular expression capability that does not provide
the power nor the flexibility UNIX programmers or users will be accustomed
to. mSQL supports the "standard" SQL regular expression
syntax, via the LIKE operator, but also provide further functionality
if it is required. The available regular expression operators are:
- LIKE - the standard SQL regular expression operator.
- CLIKE - a standard LIKE operator that ignores case.
- RLIKE - a complete UNIX regular expression operator.
Note : CLIKE and RLIKE are not standard SQL and
may not be available in other implementations of the language if you
decide to port your application. They are however very convenient and
powerful features of mSQL.
The regular expression syntax supported by the LIKE
and CLIKE operators is that of standard SQL and is outlined below
| `_' |
matches any single character |
| `%' |
matches 0 or more characters of any value |
| `\' |
escapes special characters (e.g. `\%' matches
% and `\\' matches \ ) |
| |
all other characters match themselves |
As an example of the LIKE operator, it is possible to
search for anyone in the finance department who's last name consists
of any letter followed by `ughes', such as Hughes. The query to perform
this operation could look like
SELECT first_name, last_name FROM emp_details
- WHERE dept = `finance' and last_name like `_ughes'
The RLIKE operator provides access to the power
of the UNIX standard regular expression syntax. The UNIX regular
expression syntax provides far greater functionality than SQL's
LIKE syntax. The UNIX regex syntax does not use the '_' or '%' characters
in the way SQL's regex does (as outlined above). The syntax available
in the RLIKE operator is
| '.' |
matches any single character |
| '^' |
When used as the first charactr in a regex, the
caret character forces the match to start at the first character
of the string |
| '$' |
When used as the last charactr in a regex, the
dollar sign forces the match to end at the last character of the
string |
| '[ ]' |
By enclosing a group of single characters withing
square brackets, the regex will match a single character from
the group of characters. If the ']' character is one of the characters
you wish to match you may specifiy it as the first character in
the group without closing the group (e.g. '[]abc]' would match
any single character that was either ']', 'a', 'b', or 'c'). Ranges
of characters can be specified within the group using the 'first-last'
syntax (e.g. '[a-z0-9]' would match any lower case letter or a
digit). If the first charactr of the group is the '^' character
the regex will match any single character that is not contained
within the group. |
| '*' |
If any regex element is followed by a '*' it
will match zero or more instances of the regular expression. |
The power of a relational query language starts to become
apparent when you join tables together during a select operation. Lets
say you had two tables defined, one containing staff details and another
listing the projects being worked on by each staff member, and each
staff member has been assigned an employee number that is unique to
that person. You could generate a sorted list of who was working on
what project with a query like:
SELECT emp_details.first_name, emp_details.last_name,
project_details.project
- FROM emp_details, project_details
- WHERE emp_details.emp_id = project_details.emp_id
- ORDER BY emp_details.last_name, emp_details.first_name
mSQL places no restriction on the number of tables
"joined" during a query so if there were 15 tables all
containing information related to an employee ID in some manner,
data from each of those tables could be extracted, by a single query.
One key point to note regarding joins is that you must qualify all
column names with a table name. mSQL does not support the concept
of uniquely named columns spanning multiple tables so you are forced
to qualify every column name as soon as you access more than one
table in a single select.
mSQL also supports table aliases so that you can
perform a join of a table onto itself. This may appear to be an
unusual thing to do but it is a very powerful feature if there are
rows within a single table relate to each other in some way. An
example of such a table could be a list of people including the
names of their parents. In such a table there would be multiple
rows with a parent/child relationship. Using a table alias you could
find out any grandparents contained in the table using something
like
- SELECT t1.parent, t2.child from parent_data=t1,
parent_data=t2
- where t1.child = t2.parent
The table aliases t1 and t2 both point to the same
table (parent_data in this case) and are treated as two different
tables that just happen to contain exactly the same data.
The Delete Clause
The SQL DELETE construct is used to remove one or more
entries from a database table. The selection of rows to be removed from
the table is based on the same where construct as used by the
SELECT clause. The syntax for mSQL's delete clause is
DELETE FROM table_name
- WHERE column OPERATOR value
- [ AND | OR column OPERATOR value ]**
OPERATOR can be <,>, =, <=, =, <>, LIKE, RLIKE, or CLIKE
for example
DELETE FROM emp_details WHERE emp_id = 12345
The Update Clause
The SQL update clause is used to modify data that is
already in the database. The operation is carried out on one or more
rows as specified by the where construct. The value of any number
of fields on the rows matching the where construct can be updated. mSQL
places a limitation on the operation of the update clause in that it
cannot use a column name as an update value (i.e. you cannot set the
value of one field to the current value of another field). Only literal
values may by used as an update value. The syntax supported by mSQL
is
UPDATE table_name SET column=value [ , column=value
]**
- WHERE column OPERATOR value
- [ AND | OR column OPERATOR value ]**
OPERATOR can be <,> , =, <=, =, <>, LIKE,
RLIKE or CLIKE
for example
UPDATE emp_details SET salary=30000 WHERE emp_id
= 1234
Mini SQL 2.0 (Beta) System
Variables
Introduction
Mini SQL 2.0 includes internal support for system variables
(often known as pseudo fields or pseudo columns). These variables can
be accessed in the same way that normal table fields are accessed although
the information is provided by the database engine itself rather than
being loaded from a database table. System variables are used to provide
access to server maintained information or meta data relating to the
databases.
System variables may be identified by a leading underscore
in the variables name. Such an identifier is not valid in mSQL for table
or field names. Examples of the supported system variables and uses
for those variables are provided below.
Available System Variables
The mSQL 2 engine currently supports the following system
variables:
_rowid
The _rowid system variable provides a unique row
identifier for any row in a table. The value contained in this variable
is the internal record number used by the mSQL engine to access
the table row. It may be included in any query to uniquely identify
a row in a table. An example of such queries could be :
- select _rowid, first_name, last_name from emp_details
- where last_name = 'Smith'
- update emp_details set title = 'IT Manager'
- where _rowid = 57
The candidate row module is capable of utilising _rowid
values to increase the performance of the database. In the second
example query above, only 1 row (the row with the internal record
ID of 57) would be accessed. This is in contrast to a sequential
search through the database looking for that value which may
result in only 1 row being modified but every row being accessed.
Using the _rowid value to constrain a search is the fastest
access method available in mSQL 2.0. As with all internal access
decisions, the decision to base the table access on the _rowid
value is automatic and requires no action by the programmer
or user other than including the _rowid variable in the where
clause of the query.
_timestamp
The _timestamp system variable contains the time
at which a row was last modified. The value, although specified
in the standard UNIX time format (i.e. seconds since the epoch),
is not intended for interpretation by application software. The
value is intended to be used as a point of reference via which an
application may determine if a particular row has was modified before
or after another table row. The application should not try to determine
an actual time from this value as the internal representation used
may change in a future release of mSQL.
The primary use for the _timestamp system variable
will be internal to the mSQL engine. Using this information, the
engine may determine if a row has been modified after a specified
point in time (the start of a transaction for example). It may also
use this value to synchronise a remote database for database replication.
Although neither of these functions is currently available, the
presence of a row timestamp is the first step in the implementation.
Example queries may be:
- select first_name, _timestamp from emp_details
- where first_name like '%fred%'
- order by _timestamp
- select * from emp_details
- where _timestamp 88880123
_seq
The _seq system variable is used to access the current
sequence value of the table from which it is being selected. The
current sequence value is returned and the sequence is update to
the next value in the sequence (see the CREATE section of the Language
Specification section from more information on sequences).
An example query using _seq could be
- select _seq from staff
_sysdate
The server can provide a central standard for the
current time and date. If selected from any table, the _sysdate
system variable will return the current time and date on the server
machine using the standard UNIX time format (e.g. seconds since
the epoch).
An example query using _sysdate could be
- select _sysdate from staff
_user
By selecting the _user system variable from any
table, the server will return the username of the user who submitted
the query.
An example query using _user could be
- select _user from staff
Mini SQL 2.0 (Beta) Standard
Programs and Utilities
The monitor - msql
| Usage |
msql [-h host] [-f confFile]
database |
| Options |
-h |
Specify a remote hostname
or IP address on which the mSQL server is running. The default is
to connect to a server on the localhost using a UNIX domain socket
rather than TCP/IP (which gives better performance) |
| -f |
Specify a non-default configuration
file to be loaded. The default action is to load the standard configuration
file located in INST_DIR/msql.conf (usually /usr/local/Hughes/msql.conf) |
| Description |
The mSQL monitor is an interactive
interface to the mSQL server. It allows you to submit SQL commands
directly to the server. Any valid mSQL syntax can be entered at the
prompt provided by the mSQL monitor.
Control of the monitor itself is provided by
4 internal commands. Each command is comprised of a backslash
followed by a single character. The available command are
|
|
| \q |
Quit |
|
| \g |
Go (Send the query to the server) |
| \e |
Edit (Edit the previous query) |
| \p |
Print (Print the query buffer) |
Schema viewer - relshow
| Usage |
relshow [-h host] [-f confFile]
[database [rel [idx] ] ] |
| Options |
-h |
Specify a remore hostname or IP address
on which the mSQL server is running. The default is to connect to
a server on the localhost using a UNIX domain socket rather than TCP/IP
(which gives better performance) |
| -f |
Specify a non-default configuration file
to be loaded. The default action is to load the the standard configuration
file located in INST_DIR/msql.conf (usually /usr/local/Hughes/msql.conf) |
| Description |
Relshow is used to display
the structure of the contents of mSQL databases. If no arguments are
given, relshow will list the names of the databases currently defined.
If a database name is given it will list the tables defined in that
database. If a table name is also given then it will display the structure
of the table (i.e. field names, types, lengths etc).
If an index name is provided along with the database
and table names, relshow will display the structure of the specified
index including the type of index and the fields that comprise
the index.
|
Admin program - msqladmin
| Usage |
msqladmin [-h host] [-f confFile]
[-q] Command |
| Options |
-h |
Specify a remore hostname
or IP address on which the mSQL server is running. The default is
to connect to a server on the localhost using a UNIX domain socket
rather than TCP/IP (which gives better performance) |
| -f |
Specify a non-default configuration
file to be loaded. The default action is to load the the standard
configuration file located in INST_DIR/msql.conf (usually /usr/local/Hughes/msql.conf) |
| -q |
Put msqladmin into quiet
mode. If this flag is specified, msqladmin will not prompt the user
to verify dangerous actions (such as dropping a database). |
| Description |
msqladmin is used to perform
administrative operations on an mSQL database server. Such tasks include
the creation of databases, performing server shutdowns etc. The available
commands for msqladmin are |
| create db_name |
Creates a new database called db_name |
| drop db_name |
Removes the database called db_name from
the server. This will also delete all data contained in the database!
|
| shutdown |
Terminates the mSQL server. |
| reload |
Forces the server to reload ACL information.
|
| version |
Displays version and configuration information
about the currently running server. |
| stats |
Displays server statistics. |
|
Note : most administrative functions can
only be executed by the user specified in the run-time configuration
as the admin user. They can also only be executed from the host
on which the server process is running (e.g. you cannot shutdown
a remote server process).
|
Data dumper - msqldump
| Usage |
msqldump [-h host] [-f confFile]
[-c] [-v] database [table] |
| Options |
-h |
Specify a remore hostname or IP address
on which the mSQL server is running. The default is to connect to
a server on the localhost using a UNIX domain socket rather than TCP/IP
(which gives better performance) |
| -f |
Specify a non-default configuration file
to be loaded. The default action is to load the the standard configuration
file located in INST_DIR/msql.conf (usually /usr/local/Hughes/msql.conf) |
| -c |
Include column names in INSERT commands
generated by the dump. |
| -v |
Run in verbose mode. This will display
details such as connection results etc. |
| Description |
msqldump produces an ASCII
text file containing valid SQL commands that will recreate the table
or database dumped when piped through the mSQL monitor program. The
output will include all CREATE TABLE commands required to recreate
the table structures, CREATE INDEX commands to recreate the indices,
and INSERT commands to populate the tables with the data currently
contained in the tables.
Note : msqldump does not recreate sequences
at this time.
|
Data exporter - msqlexport
| Usage |
msqlexport [-h host] [-f
conf] [-v] [-s Char] [-q Char] [-e Char] database table |
| Options |
-h |
Specify a remore hostname or IP address
on which the mSQL server is running. The default is to connect to
a server on the localhost using a UNIX domain socket rather than TCP/IP
(which gives better performance) |
| -f |
Specify a non-default configuration file
to be loaded. The default action is to load the the standard configuration
file located in INST_DIR/msql.conf (usually /usr/local/Hughes/msql.conf) |
| -v |
Verbose mode |
| -s |
Use the character Char as the separation
character. The default is a comma. |
| -q |
Quote each value with the specified character |
| -e |
Use the specifed Char as the escape character.
The default is \ |
| Description |
msqlexport produces an ASCII
export of the data from the specified table. The output produced can
be used as input to other programs such as spreadsheets. It has been
designed to be as flexible as possible allowing the user to specify
the character to use to separate the fields, the character to use
to escape the separator character if it appears in the data, and whether
the data should be quoted and if so what character to use as the quote
character.
The output is sent to stdout with one data row
per line.
|
Data importer - msqlimport
| Usage |
msqlimport [-h host] [-f
conf] [-v] [-s Char] [-e Char] [-c col,col...] database table |
| Options |
-h |
Specify a remore hostname or IP address
on which the mSQL server is running. The default is to connect to
a server on the localhost using a UNIX domain socket rather than TCP/IP
(which gives better performance) |
| -f |
Specify a non-default configuration file
to be loaded. The default action is to load the the standard configuration
file located in INST_DIR/msql.conf (usually /usr/local/Hughes/msql.conf) |
| -v |
Verbose mode |
| -s |
Use the character Char as the separation
character. The default is a comma. |
| -e |
Use the specifed Char as the escape character.
The default is \ |
| -c |
A comma separated list of column names
into which the data will be inserted.
Note : there can be no spaces in the list. |
| Description |
msqlimport loads a flat ASCII
data file into an mSQL database table. The file can be formatted using
any character as the column separator. When passed through msqlimport,
each line of the txt file will be loaded as a row in the database
table. The separation character as specified by the -s flag, will
be used to split the line of text into columns. If the data uses a
specific character to escape any occurence of the separation character
in the data, the escape character can be specified with the -e flag
and will be removed from the data before it is inserted. |
Mini SQL 2.0 (Beta) Run Time
Configuration
Introduction
mSQL 1.x offered several configuration options, including
such details as the user the server should run as, the location of the
TCP and UNIX sockets for client/server communications, the location
of the database files etc. The problem with configuring mSQL 1.x was
that all these details were hard-coded into the software at compile
time. Once the software was compiled and installed you couldn't easily
change those settings.
To overcome this problem, mSQL 2.0 utilises an external
run-time configuration file for definition of all these values. The
file is called msql.conf and is located in the installation
directory (usually /usr/local/Hughes). An application can choose to
use a different configuration file by calling the new msqlLoadConfigFile(
) API function. All standard mSQL applications and utilities
provide a command line flag, -f ConfFile , that
allows you to specify a non-standard configuration file. When an application
first calls the mSQL API library, a check is made to see if a configuration
file has been loaded via a call to the msqlLoadConfigFile( ) function.
If no such call has been made, the API library loads the default config
file. Any values that are specified in that file will over-ride the
normal operating paramaters used by mSQL.
Structure of the config file
The configuration file is a plain text file organised
into sections. The file can contain blank lines and comments. A comment
is a line that begins with the '#' character. Each section of the configuration
file has a section header, which is written as the section name enclosed
in square brackets (for example [ general ]). Currently the only
section defined is the general section although further sections
covering security and access control will be added later.
Configuration values within a section are presented
using the config parameter name followed by and equals sign and then
the new value. There can only be one entry per line and if an entry
is defined multiple times in the one config file the last value defined
will be used. If a parameter is not defined in the config file then
an internal default value will be used at run-time.
Elements of the General section
The following configuration parameters are available
in the general section of the config file. Please note that %I
may be used in configuration entries to signify the mSQL installation
directory (e.g. /usr/local/Hughes).
|
Parameter
|
Default Value
|
Definition
|
| Inst_Dir |
/usr/local/Hughes |
The full path to the installation directory.
This is the directory in which all the mSQL files are located
(such as the program files, the database files etc). |
| mSQL_User |
msql |
The user that the mSQL server should run as.
If the server is started by a user other than this user (e.g.
it is started as root from a boot script) it will change UID so
that it runs as the specified user. |
| Admin_User |
root |
The user that is allowed to perform privileged
operations such as server shutdown, cration of databases etc.
|
| Pid_File |
%I/msql2.pid |
The full path of a file in which the PID of the
running mSQL server process will be stored. |
| TCP_Port |
1114 |
The TCP port number on which the mSQL server
will accept client/server connections over a TCP/IP network. If
this value is modified it must be modified on the machine running
the client software also. |
| UNIX_Port |
%I/msql2.sock |
The full path name of the UNIX domain socket
created by the mSQL server for connections from client applications
running on the same machine. |
Example configuration file
Below is a sample configuration file. This file does
not achieve anything as it just sets the parameters to their default
values.
#
# msql.conf - Configuration file for Mini SQL Version 2
#
#--------------------------------------------------------------
#
# This file is an example configuration and may require
# modification to suit your needs or your site. The values
# given are the default values and will be used by the
# software if either this file is missing or a specific value
# is not specified.
#
#--------------------------------------------------------------
[general]
Inst_Dir = /usr/local/Hughes
mSQL_User = msql
Admin_User = root
Pid_File = %I/msql2.pid
TCP_Port = 1114
UNIX_Port = %I/msql2.sock
|
|