Page 1 of 2

FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sat Mar 25, 2017 9:20 am
by nageswaragunupudi
New RecSet class is under development for loading and browsing very large tables. For testing and development, we are using a free cloud server which is obviously slower than local servers or business class servers. In version 17.03 this class is available for use with a limited functionality and will be fully functional in 17.04.

In this post we present a preview of the new class. We have a table `custbig` on the cloud server with one million records. This table is created by inserting "customer.dbf" 2000 times.

This is sample program. ( fwh\samples\maria10.prg )

Code: Select all

#include "fivewin.ch"

function Main()

   local oCn, oRs, nRecs

   oCn   := FW_DemoDB()
   ? nRecs := oCn:QueryResult( "SELECT COUNT(*) FROM custbig" ), "Record in table `custbig`"
   oRs   := oCn:RecSet( "custbig", 1000 )
   oRs:nLastRec   := nRecs
   XBROWSER oRs TITLE "`Custbig` table with million records. Ready to browse in " + ;
      Transform( oRs:nReadSecs, "999.9999" ) + " Seconds"  SHOW RECID FASTEDIT
   oCn:Close()

return nil
 
Image

After you download FWH17.03 you can build and test this program by executing
build? maria10
You do not need to have access to any mysql/mariadb server to run this program. This program will connect to the cloud server and display the table.

Even if you do not have FWH17.03 or even if you do not have FWH at all, still you can test this program by downloading zip file from the link below.

http://anserkk.com/gnraomysql/view.php?id=1

Download the zip file, unzip the contents into any blank folder, switch to that folder and run maria10.exe from that folder.

Depending on your own internet speeds, normally you should see the browse of million records within about one second and will be able to gobottom, gotop or use vertical scrollbar for moving to anywhere in the table very fast. You can also modify the data by inline edit or edit dialog and check the performance. We disabled appends in this test program.

Please note that we are getting from a free cloud based server and performance in real life on business -servers / inhouse-servers should be far better.

Whether you are interested or not in FWMARIADB and whatever version of FWH you are using, we encourage you download the EXE and run it and experience it.

We welcome your feedback as well as criticism.

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sat Mar 25, 2017 9:45 am
by kok joek hoa
Dear G. N. Rao.,

just 1.89 second for get all data

Regards,

Kok

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sat Mar 25, 2017 11:01 am
by dpesic
Thats realy amazing ! :)

Is there any chance for 'FWPOSTGRESDB' ? :)

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sat Mar 25, 2017 11:26 am
by nageswaragunupudi
Is there any chance for 'FWPOSTGRESDB' ? :)
As of now, we do not have any plans.
We would consider if we see many users interested in it.

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sat Mar 25, 2017 12:37 pm
by miragerr
Hello, friends

I'm very interested in PostGreSql, including can collaborate in the development.

I'm standing by.

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sat Mar 25, 2017 12:55 pm
by dpesic
nageswaragunupudi wrote:
Is there any chance for 'FWPOSTGRESDB' ? :)
As of now, we do not have any plans.
We would consider if we see many users interested in it.
How many is minimum :) ? or ... maybe minimum budget for this ?

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sat Mar 25, 2017 2:36 pm
by nageswaragunupudi
I see that (x)Harbour contributions already have postgre library. I have never used though.

May we know what more our members are expecting?

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sun Mar 26, 2017 3:47 pm
by MGA
Parabéns pelo excelente trabalho.

Sr. Nages, todas estas funcionalidades estão disponíveis tanto para MYSQL como para MARIADB?

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sun Mar 26, 2017 3:55 pm
by nageswaragunupudi
MYSQL como para MARIADB?
Yes.
Both MariaDB server and MySql server behave the same way.
FWH application whether linked with libmariadb.lib or libmysql.lib and run with libmariadb.dll or libmysql.dll work the same way both on mariadb server and mysql server.

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Sun Mar 26, 2017 9:42 pm
by James Bott
Wow!

Color me impressed.

James

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Mon Mar 27, 2017 5:37 am
by hua
nageswaragunupudi wrote:I see that (x)Harbour contributions already have postgre library. I have never used though.

May we know what more our members are expecting?
I am interested in Firebird.

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Mon Mar 27, 2017 7:32 am
by Maurizio
Hello Rao.,

just 0.89 for the first block

Regards,

Maurizio

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Mon Mar 27, 2017 8:25 am
by Enrico Maria Giordano
Wow, 0.28 seconds here! :-)

Great work, Rao!

EMG

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Mon Mar 27, 2017 11:11 am
by dutch
here is 1.023 seconds, great job.

Re: FWMARIADB 17.03 Fast Loading/Browsing Very Large Tables

Posted: Mon Mar 27, 2017 12:22 pm
by Marcelo Via Giglio
PostgreSQL +1