Русская версия English version  
Documentation - NVStat for NetWare

The loader functionally consists of 2 script groups:
1-st group: Scripts, written on Perl 5.8 language - intended for log files processing and placing information into MySQL database.
2-nd group: Scripts, written on PHP language - intended for statistics viewing through any Web browser.

System requirements:
	
Platform:		*nux/Netware 6.x/WinNT/2000/2003
Language:		Perl 5.8 with additional PM modules, viewer written and tested on PHP 4.x
Database engine:	MySQL v.3 and higher

Installation and operation:

1. Check the NetWare server for presence of following packages:
	a) latest version of LibC (http://developer.novell.com/ndk/libc.htm)
	b) latest version of Novell Script for NetWare (http://developer.novell.com/ndk/nscript.htm)
	c) latest version of Universal Component System (http://developer.novell.com/ndk/ucs.htm)
	d) latest version of LDAP libraries for C (http://developer.novell.com/ndk/cldap.htm)
	e) latest version of Perl 5.8 for NetWare (http://developer.novell.com/ndk/perl5.htm)
	f) latest version of PHP for NetWare (http://developer.novell.com/ndk/php.htm)
	g) latest version of MySQL 4.0.x for NetWare (http://dev.mysql.com/downloads/mysql/4.0.html)

2. Copy archive containment to the server, which corresponds to system requirements.
   a) Copy PERL folder to the SYS volume;
   b) Copy NVSTAT folder to the Web server root directory;
   c) Copy SYSTEM folder to the SYS volume.

3. Verify next parameter in Border Manager configuration file (sys:etc/proxy/proxy.cfg):
   [Log Format]
   It must have the value: Delimiter-Character=space

4. Execute script for creation of necessary tables:
SYS:PERL/WEB/NVSTAT/SQL/nvstat.sql - Border Manager 3.X/WanStat/TSE 0.93f database without indexes;
SYS:PERL/WEB/NVSTAT/SQL/nvstat_wi.sql - Border Manager 3.X/WanStat/TSE 0.93f database with indexes;
SYS:PERL/WEB/NVSTAT/SQL/nvstat_index.sql - creates indexes in the existing database Border Manager 3.X/WanStat/TSE 0.93f;
SYS:PERL/WEB/NVSTAT/SQL/ics.sql - Volera Excelerator database without indexes.

I suppose, that you know all advantages and disadvantages of indexed and not-indexed database.
The selection from indexed database is faster, but pasting is rather slower.
In the not-indexed database the pasting is fast, but selection is slower, and often it is more slower.

We recommend to use EMS MySQL Manager 2 for manipulations with databases (http://ems-hitech.com/mymanager/).

5. Edit file SYS:PERL/WEB/NVSTAT/nvstat.conf, according to your system settings and your requirements.

    Brief description of the nvstat.conf file configuration parameters:


    a) BorderManager log loader settings
    # HTTP statistics settings

    MySQL server address
    http_sql_host 127.0.0.1

    User name for access to the database
    http_sql_user NVStat

    Password
    http_sql_pass stat

    Database name
    http_sql_base NVStat

    Surfing statistics table name
    http_sql_proxy_tbl proxy_stat

    Large files download statistics table name 
    http_sql_files_tbl files_stat

    Full statistics table name (the load is getting enabled by http_dumpfull parameter)
    http_sql_full_tbl full_proxy_stat 

    COMMON log placement
    http_common_log_dir sys:/etc/proxy/log/HTTP/Common/

    EXTENDED log placement (IMPORTANT !!! EXTENDED log is mandatory !!!!)
    http_extended_log_dir sys:/etc/proxy/log/HTTP/Extended/

    COMMON log file name mask %year%month%day (IMPORTANT !!! The program is designed for work with log, generated by Border Manager with selected parameter "Rollover Every 1 days")
    http_common_log_mask %y%m%d.LOG

    EXTENDED log file name mask %year%month%day (IMPORTANT !!! The program is designed for work with log, generated by Border Manager with selected parameter "Rollover Every 1 days")
    http_extended_log_mask %y%m%d.LOG

    Information about loading: "file name" 
    http_position_file sys:/perl/web/nvstat/http.pos

    Minimum size of the file, registering in "large files" download table (in Bytes)
    http_critical_mass 500000
    
    It is necessary, that the script would automatically load piece of log for previous day.
    For example, if we execute the script at 23:10 and then at 00:10, so, the log will be loaded only for current day.
    Parameter "auto_prevday" enables checking for the current time and if it is less than 00:15:00, so script must load additional portion of log for the previous day.
    http_auto_prevday yes

    Loader working log placement
    http_logfile sys:/perl/web/nvstat/proxy.log

    Removal of leading period in the user's name (IMPORTANT!!! You must set this parameter to "yes", otherwise viewer (on PHP) won't show anything to you).
    http_dropdot yes

    Full log enabling, i.e. refusal of information aggregation during the process of data loading
    (for example, in an interval between executions of this script, you have downloaded 2 files (10 KBytes each) with this parameter disabled.
    One line, which says, that you have got 20 KBytes from this site, will be written into log, with date and time corresponding to the script execution time.)
    http_dumpfull no

    Parameter, specifying, that only lines with specified HTTP codes will be selected for placing to the database. (codes are separated by commas, i.e. 200,403)
    In this example all HTTP codes are included, but you may exclude some codes, if you want. Be careful - some codes are very importatnt, e.g. 200 and 206 !!!
    http_only_codes 100,101,200,201,202,203,204,206,300,301,302,303,304,305,307,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,500,501,502,503,504,505
    
    Parameter, specifying, that only lines, with HTTP data, received from cache will be processed. Default is no.
    http_only_noncached no
    
    Parameter, specifying list of ignored users. Users are separated by commas. (IMPORTANT!!! User names are case-sensitive!)
    For example: http_ignore_users - or http_ignore_users -,victor.ZGPU
    http_ignore_users -
    
    Parameter, specifying list of ignored hosts. For example: http_ignore_url novell.chel.ru,192.168.95.2
    Only beginning of the URL will be used, i.e.: http://(novell.chel.ru):8100/ (only full qualified host name in the brackets will be used)
    http_ignore_url novell.chel.ru 

    Parameter, specifying list of "non-authorized users"-to-"IP address" translations
    http_userip_aliases 192.168.1.10:Vasya;192.168.1.11:Lenochka


    b) Volera Excelerator log loader settings
    # Volera statistics settings

    MySQL server address
    ics_sql_host 127.0.0.1

    User name for access to the database
    ics_sql_user NVStat

    Password
    ics_sql_pass stat

    Database name
    ics_sql_base NVStat

    Surfing statistics table name
    ics_sql_proxy_tbl ics_stat

    Large files download statistics table name 
    ics_sql_files_tbl files_ics_stat

    Full statistics table name (the load is getting enabled by ics_dumpfull parameter)
    ics_sql_full_tbl full_ics_stat 

    Volera Excelerator log placement
    ics_log_dir ../ICS/

    Volera log file name mask %year%month%day
    ics_log_mask %y%m%d.LOG

    Information about loading: "file name" 
    ics_position_file sys:/perl/web/nvstat/ics.pos

    Minimum size of the file, registering in "large files" download table (in Bytes)
    ics_critical_mass 500000
    
    It is necessary, that the script would automatically load piece of log for previous day.
    For example, if we execute the script at 23:10 and then at 00:10, so, the log will be loaded only for current day.
    Parameter "auto_prevday" enables checking for the current time and if it is less than 00:15:00, so script must load additional portion of log for the previous day.
    ics_auto_prevday yes

    Loader working log placement
    ics_logfile sys:/perl/web/nvstat/ics.log

    Removal of leading period in the user's name (IMPORTANT!!! You must set this parameter to "yes", otherwise viewer (on PHP) won't show anything to you).
    ics_dropdot yes

    Full log enabling, i.e. refusal of information aggregation during the process of data loading
    (for example, in an interval between executions of this script, you have downloaded 2 files (10 KBytes each) with this parameter disabled.
    One line, which says, that you have got 20 KBytes from this site, will be written into log, with date and time corresponding to the script execution time.)
    ics_dumpfull no

    Parameter, specifying, that only lines with specified HTTP codes will be selected for placing to the database. (codes are separated by commas, i.e. 200,403)
    In this example all HTTP codes are included, but you may exclude some codes, if you want. Be careful - some codes are very importatnt, e.g. 200 and 206 !!!
    ics_only_codes 100,101,200,201,202,203,204,206,300,301,302,303,304,305,307,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,500,501,502,503,504,505
    
    Parameter, specifying, that only lines, with HTTP data, received from cache will be processed. Default is no.
    ics_only_noncached no
    
    Parameter, specifying list of ignored users. Users are separated by commas. (IMPORTANT!!! User names are case-sensitive!)
    For example: ics_ignore_users - or ics_ignore_users -,victor.ZGPU
    ics_ignore_users -
    
    Parameter, specifying list of ignored hosts. For example: ics_ignore_url novell.chel.ru,192.168.95.2
    Only beginning of the URL will be used, i.e.: http://(novell.chel.ru):8100/ (only full qualified host name in the brackets will be used)
    ics_ignore_url novell.chel.ru 

    Parameter, specifying list of "non-authorized users"-to-"IP address" translations
    ics_userip_aliases 192.168.1.10:Vasya;192.168.1.11:Lenochka

    c) Traffic Shaper log loader settings (www.trafficshaper.com)
    # Shaper statistics settings

    MySQL server address
    shaper_sql_host 127.0.0.1

    User name for access to the database
    shaper_sql_user NVStat

    Password
    shaper_sql_pass stat

    Database name
    shaper_sql_base NVStat
    
    Traffic shaper statistics table name
    shaper_sql_table shaper_stat
    
    Shaper log placement
    shaper_log_dir sys:/
    
    Shaper log file name mask
    shaper_log_mask SHAPER.ADT
    
    Information about loading: "file name" 
    shaper_position_file sys:/perl/web/nvstat/shaper.pos

    It is necessary, that the script would automatically load piece of log for previous day.
    For example, if we execute the script at 23:10 and then at 00:10, so, the log will be loaded only for current day.
    Parameter "auto_prevday" enables checking for the current time and if it is less than 00:15:00, so script must load additional portion of log for the previous day.
    shaper_auto_prevday yes

    Loader working log placement
    shaper_logfile sys:/perl/web/nvstat/shaper.log


    d) WANstat log loader settings (www.officenet.hu/utils/wanstat/index.htm)
    # WAN statistica settings

    MySQL server address
    wan_sql_host 127.0.0.1
    
    User name for database access
    wan_sql_user NVStat

    Password
    wan_sql_pass stat
    
    Database name
    wan_sql_base NVStat

    WANstat statistics table name
    wan_sql_table wan_stat
    
    Last WANstat statistics table name
    wan_sql_table_last wan_stat_last
    
    WANstat log files placement
    wan_log_dir sys:/etc/Wanstat/

    WANstat log file mask
    wan_log_mask %Y%m%d.TXT

    Information about loading: "file name" 
    wan_position_file sys:/perl/web/nvstat/wan.pos

   It is necessary, that the script would automatically load piece of log for previous day.
   For example, if we execute the script at 23:10 and then at 00:10, so, the log will be loaded only for current day.
   Parameter "auto_prevday" enables checking for the current time and if it is less than 00:15:00, so script must load additional portion of log for the previous day.
   wan_auto_prevday yes
    
    Loader working log placement
    wan_logfile sys:/perl/web/nvstat/wan.log 

    WAN server name (may be useful, if you have more, than one WANstat server)
     wan_NAS CHRT

6. Execute the necessary script:

For example:

    a) perl sys:/perl/web/nvstat/http_rl.pl prev --noscreen
       (log loading for the current day)
    
    b) perl sys:/perl/web/nvstat/http_rl.pl import "2003-03-15" "2003-07-16"
       (log loading for the specified time interval)

For automation of statistics gathering process it is possible to use standard scheduler CRON. To do this, add a line into SYS:ETC/crontab file:

10,20,30,40,50,0    *    *    *    0,1,2,3,4,5,6    bmlogday.ncf

According to this line, the SYS:SYSTEM/bmlogday.ncf file will be executed every 10 minutes.
bmlogday.ncf file contains the next line:

perl sys:perl/web/nvstat/http_rl.pl prev --noscreen

Don't forget to add CRON.NLM to SYS:SYSTEM/autoexec.ncf file for automatic starting of scheduler at the server startup!

7. Viewer configuration
   
   a) Create Organizational Unit NVStat
       Create 3 groups inside NVStat: NVStat-Admin-Allow, NVStat-User-Deny and NVStat-Buh-Allow.
       Purpose of these groups will be discussed later, at the description of the parameter $M_Login.
       Attention!!! Keep case of characters!!!

       There is a new function, defined in the new version of NVStat: "Quotas"
       Purpose of this function - group-based control of Internet using.
       For use this function you need to create some Groups inside the NVStat OU.

       Group naming format:
       	Quote-XXX, where XXX - quotas in megabytes.

       For example:
       	Quote-10 - 10-MBytes quota,
       	Quote-50 - 50-MBytes quota,
       	Quote-500 - 500-MBytes quota.

        You may add description for these groups by using attribute "Other Name" in Object Property page.

        For example:
        	Quote-500 - "Other Name": Quota for advanced users.

        Example of NDS structure:

	CHRT
	 |
	 |-----NVStat
	 |	    |--NVStat-Admin-Allow
	 |	    |--NVStat-Buh-Allow
	 |	    |--NVStat-User-Deny
	 |	    |--Quote-10
	 |	    |--Quote-50
	 |	    |--Quote-100
	 |	    |--Quote-200
	 |
	 |-----UC
	 |	|--
	 .	.
	 .	.
   
   
   Remarks to steps (b),(c),(d):
   You may use predefined file php.ini from NVStat package and skip the steps (b),(c),(d).
   Take it from PHP directory, copy to the SYS:/PHP/ directory and restart the Apache.

   b) For PHP on NetWare platform, edit file SYS:PHP/php.ini
       Find the error_reporting parameter and change its value to E_ALL & ~E_NOTICE & ~E_WARNING
       (error_reporting= E_ALL & ~E_NOTICE & ~E_WARNING)
   
   c) For PHP on NetWare platform, edit file SYS:PHP/php.ini
      Find the default_charset parameter and change its value to windows-1251
      (default_charset = "windows-1251")

   d) For PHP on NetWare platform, comment out "open_basedir" parameter in the SYS:PHP/php.ini file.

   e) For PHP on NetWare platform, edit file SYS:PHP/php.ini
      Find the register_globals parameter and change its value to On
      (register_globals = On)

   f) Edit file /NVSTAT/config.php, according to your system settings and your requirements.

   Brief description of the config.php file configuration parameters:

   MySQL server IP address
   $SQL_Host = "127.0.0.1";

   Database user name
   $SQL_User = "NVStat";

   Database user password
   $SQL_Passwd = "stat";

   Database name
   $SQL_Base = "NVStat";
   
   Proxy statistics table name
   $SQL_Proxy_Stat_Tbl = "proxy_stat";

   Files statistics table name
   $SQL_Files_Stat_Tbl = "files_stat";

   Shaper statistics table name
   $SQL_Shaper_Stat_Tbl = "shaper_stat";

   WAN statistics table name
   $SQL_Wan_Stat_Tbl = "wan_stat";

   Display statistics in: 0-Bytes, 1-KBytes, 2-MBytes, 3-GBytes
   Remark: this parameter is active only for "Proxy"->"Personal" tab.
   $ReSize = "2";
   
   To hide display of file statistics ( 1 - Hide, 0 - Display )
   Remark: this parameter is active only for "Proxy"->"Personal" tab.
   $HideFiles = "0";
   
   To hide display "Downloaded from Internet" and "Taken from cache" ( 1 - Hide, 0 - Display )
   Remark: this parameter is active only for "Proxy"->"Personal" tab.
   $HideINet = "0";
   
   To hide display of TOP20 ( 1 - Hide, 0 - Display )
   Remark: this parameter is active only for "Proxy"->"Personal" tab.
   $HideTop20 = "0";

   Authorization method ( 0 - LDAP, 1 - OCX )
   ATTENTION!!! We strongly recommend to use LDAP authorization, because OCX or Transparent Authorization has a number of restrictions, namely:
   - Using Novell Client 32 is mandatory;
   - At this time statistics access restriction functions has not been realized yet. (they will be implemented in the next versions of NVStat)
   So, at this time, statistics viewing via OCX is possible with User rights or Admin rights for all users (see parameter $OCX_Status).
   But there is also significant plus in using OCX authorization - it isn't need to enter login name and password.
   That is, at the statistics view page, User Name and Context will be automatically detected.
   ATTENTION!!! To do the LDAP authorization to work properly, it is necessary to do some actions:
   a) Check up that the [Public] object was the trustee of Organization object, where LDAP server is installed and had Object Rights - 
       Browse, Inheritable and Property Rights - Compare, Read, Inheritable.
       If those rights aren't set, you won't authorize on LDAP server.
   b) If you use Novell eGuide, that for certain you created a special proxy-user (for example: "Anonymous") and, 
        in "LDAP Group -> Authentication Options -> Proxy User" settings you entered exactly "Anonymous".
        Then, check up that this user was the trustee of the Organization object, where LDAP server is installed and had all rights, described just above.
        For viewing of LDAP server settings use ConsoleOne or iManager.
   It is necessary to note that User and Admin rights means:

   User Right - absolutely all users in the tree have this right; it gives to users viewing only they own Internet access statistics and nobody else; 
   it is possible to view statistics for the current month or for any month.
   If you don't want that any selected user has got access to the statistics - it is enough to add this user into a group NVStat-User-Deny.

   Buh Right - is an appointed right. To receive it, it is necessary to add the selected user into a group NVStat-Buh-Allow.
   User with this right can view both of his own statistics and statistics of any user

   Admin Right - is an appointed right. To receive it, it is necessary to add the selected user into a group NVStat-Admin-Allow.
   User with this right can view both of his own statistics and statistics of any user; also he has the right to view Shaper statistics and WAN statistics.
   Also it is necessary to note, that deny right has higher priority, than Admin Right.
   That is, if you have Admin status and you are the member of group NVStat-User-Deny, so you won't view statistics.
   $M_Login = "0";
   
   Access level for OCX authorization (User or Admin). ATTENTION!!! This access level is appointed absolutely to all users in a tree!
   $OCX_Status = "User";

   LDAP server IP address
   $LDAP_Host = "127.0.0.1";
   
   LDAP server context
   $LDAP_BDN = "o=CHRT";

   NVStat context
   $LDAP_BDN_Quote = "o=CHRT";

   LDAP Proxy-User
   $LDAP_BDN_User = "cn=Proxy,o=CHRT";

   LDAP Proxy-User password
   $LDAP_BDN_User_Passwd = "PASSWORD";

   Allow user password change (0 - Deny, 1 - Allow)
   $Allow_Password_Function = "0";

   Users group, for which the Administrative access is allowed
   $LDAP_Admin_Allow = array('cn=NVStat-Admin-Allow,ou=NVStat,o=CHRT');

   Users group, for which the access for viewing reports is allowed
   $LDAP_Buh_Allow = array('cn=NVStat-Buh-Allow,ou=NVStat,o=CHRT');

   Users group, for which the viewing access is denied
   $LDAP_User_Deny = array('cn=NVStat-User-Deny,ou=NVStat,o=CHRT');

   The user who can manage Internet access.
   The above user must be given permissions to manage user groups.
   In the LDAP database he must be given the following permissions:
   1. Full access to "groupMembership" and "securityEquals" attributes of $LDAP_BM_Block_Group
   2. Full access to  "member" and "equivalentToMe" attributes of NDS users.
   $LDAP_BM_AdminUser = "cn=Admin,o=CHRT";

   Internet administrator password.
   $LDAP_BM_AdminUserPasswd = "12345";  

   Table colours
   $Proxy_TreeTbl_Color = "#D0D0EA";
   $Proxy_StatHeadTbl_Color = "#C0C0C0";
   $Proxy_StatBodyTbl_Color = "#808080";
   $Proxy_StatTreeTbl_Color = "#F0F0F0";
   $Proxy_StatBackground_Color = "#E0E0E0";
   
   Auto select viewer Language (0 - Disable, 1 - Enable)
   $NVAutoLanguage = "1";

   Default viewer Language (russian or english)
   $NVLanguage = "russian";

   HTTP code selection for statistics calculation (you may specify either only one code, or, if you leave this parameter empty, calculation will be processed for all codes, specified in the nvstat.conf file, in the http_only_codes parameter)
   $NV_Code="";

   Allow MultiQuote (1 - Enable, 0 - Disable)
   $NV_MultiQuote = "0";

   Subnet, for which the access for viewing reports is allowed
   $Allow_Subnet = array('All');

   IP address, for which the access for viewing reports is allowed
   $Allow_IPAddress = array('All');

8. PHP-GD
   Download -> http://www.gknw.de/phpbb/

9. PHP-PEAR
   Download -> http://pear.php.net/package/PEAR

--------------------------------------------------------------------------------

   Congratulations!!!
   NVStat installation and configuration is finished.
   
   For viewing statistics use any Web browser.
   Go to page http:///NVStat/ and you can view the statistics.
   
   Have a nice work!!! :-)))
--------------------------------------------------------------------------------

Translated to english by Alexander Mitroshin (aka Alex-M),
Ministry of Foreign Affairs of Russia. alex-m@mid.ru

Powered by NVStat Team © 2001 - 2016 Michael Grigorev