Discussion:
Slow answer of select when using lookup fields
(too old to reply)
pcerdaz
2007-01-28 22:06:01 UTC
Permalink
I have a TIBDataSet with a SelectSQL that work’s very well,
however, when I add a lookup field to retrieve data from
another table, the answer of the SelectSQL is very slow.

The lookup field is working well but it is affecting the speed
of SelectSQL; I need this lookup field to show the information
on a Grid.

Is there another method to do it without affecting the speed of
SelectSQL?

Thanks,
pcerdaz
unknown
2007-01-28 23:20:36 UTC
Permalink
Post by pcerdaz
I have a TIBDataSet with a SelectSQL that work’s very well,
however, when I add a lookup field to retrieve data from
another table, the answer of the SelectSQL is very slow.
[snip]

Hi pcerdaz,

if you have only to retrieve data (read only), is better
to get rid of lookup fields. They possibly generates a query for each
row so to slow down your visualization. Try to replace lookup fields with
field coming from a join, i.e. modify your query in order to involve more
than one table: it should to be faster.
However, make sure you have all the primary keys defined in your lookup
tables. Having all the primary keys defined means that you have indexes
which should turn from linear to logarithmic all the accesses to lookup
tables.
Even, check if you have created correctly all the pk-fk constraints:
they can help the optimizer.

HTH

Regards

Giuliano
pcerdaz
2007-02-04 00:45:09 UTC
Permalink
Hi Giuliano,

Very good idea; I just joined the tables and now I have a
runtime query that give me those fields that I need, however
the IBDataSet where I made the SelectSQL with my query don’t
have those joined fields as part of its persistent fields from
where I can use it to show over a DBGrid. How can I do that?

Regards,
pcerdaz
Post by unknown
Post by pcerdaz
I have a TIBDataSet with a SelectSQL that work’s very well,
however, when I add a lookup field to retrieve data from
another table, the answer of the SelectSQL is very slow.
[snip]
Hi pcerdaz,
if you have only to retrieve data (read only), is better
to get rid of lookup fields. They possibly generates a query for each
row so to slow down your visualization. Try to replace lookup fields with
field coming from a join, i.e. modify your query in order to involve more
than one table: it should to be faster.
However, make sure you have all the primary keys defined in your lookup
tables. Having all the primary keys defined means that you have indexes
which should turn from linear to logarithmic all the accesses to lookup
tables.
they can help the optimizer.
HTH
Regards
Giuliano
Wayne Niddery [TeamB]
2007-02-04 03:04:55 UTC
Permalink
Post by pcerdaz
Very good idea; I just joined the tables and now I have a
runtime query that give me those fields that I need, however
the IBDataSet where I made the SelectSQL with my query don’t
have those joined fields as part of its persistent fields from
where I can use it to show over a DBGrid. How can I do that?
Just bring up the fields editor and add them - they should display in the
list when you click Add.
--
Wayne Niddery - Winwright, Inc (www.winwright.ca)
"Those who disdain wealth as a worthy goal for an individual or a
society seem not to realize that wealth is the only thing that can
prevent poverty." - Thomas Sowell
pcerdaz
2007-02-04 18:49:02 UTC
Permalink
The fields that appear in the fields editor does not include
those that I create on runtime. This is because the SelectSQL in desing time is not equal to the SelectSQL in runtime. I'm looking for a method to add a field on runtime.

Any idea?

Thank you,
pcerdaz
Post by Wayne Niddery [TeamB]
Post by pcerdaz
Very good idea; I just joined the tables and now I have a
runtime query that give me those fields that I need, however
the IBDataSet where I made the SelectSQL with my query don’t
have those joined fields as part of its persistent fields from
where I can use it to show over a DBGrid. How can I do that?
Just bring up the fields editor and add them - they should display in the
list when you click Add.
--
Wayne Niddery - Winwright, Inc (www.winwright.ca)
"Those who disdain wealth as a worthy goal for an individual or a
society seem not to realize that wealth is the only thing that can
prevent poverty." - Thomas Sowell
Wayne Niddery [TeamB]
2007-02-05 03:08:50 UTC
Permalink
Post by pcerdaz
The fields that appear in the fields editor does not include
those that I create on runtime. This is because the SelectSQL in
desing time is not equal to the SelectSQL in runtime. I'm looking for
a method to add a field on runtime.
In that case you are best not to create *any* at design time, then when you
open the query at runtime, *all* selected fields will have a TField object
created for you. Tou can get at any field at runtime with
IBDataset.FieldByName('fieldname').
--
Wayne Niddery - Winwright, Inc (www.winwright.ca)
"It is error alone which needs the support of government. Truth can
stand by itself." - Thomas Jefferson
unknown
2007-02-05 19:56:00 UTC
Permalink
Post by pcerdaz
The fields that appear in the fields editor does not include
those that I create on runtime. This is because the SelectSQL in desing time is not equal to the SelectSQL in runtime. I'm looking for a method to add a field on runtime.
Hi pcerdaz,

I'm wondering the reasons because you can't to have the same query at
design time like at runtime.

Ciao

Giuliano
pcerdaz
2007-02-06 13:57:05 UTC
Permalink
At desing time I use the SQL Editor that create automatically somthing like "select * from MI_TABLE" but at runtime I need some filters like "select * from MI_TABLE where A = B". Thinking again, probably it is posible to add the joined fields no matter the filters that I applied later.

Regards,
pcerdaz
Post by unknown
Post by pcerdaz
The fields that appear in the fields editor does not include
those that I create on runtime. This is because the SelectSQL in desing time is not equal to the SelectSQL in runtime. I'm looking for a method to add a field on runtime.
Hi pcerdaz,
I'm wondering the reasons because you can't to have the same query at
design time like at runtime.
Ciao
Giuliano
unknown
2007-02-06 15:29:53 UTC
Permalink
Post by pcerdaz
At desing time I use the SQL Editor that create automatically somthing like "select * from MI_TABLE"
but at runtime I need some filters like "select * from MI_TABLE where A = B". Thinking again,
probably it is posible to add the joined fields no matter the filters that I applied later.
Exactly, you can place a "joined" query as well.

Usually I'm starting with a no-fetch query, for example, like the following:

select o.order_no, o.order_date, c.cust_name
from orders o, customers c
where o.cust_no = c.cust_no
and 1 = 0

The line that contains "and 1 = 0" forms a no-fetch query, as the
expression 1 = 0 is always false (contraddiction or antitautology.)

A such query is used to permit to the DataSet component to retrieve
only the field definizions without loading the network to fetch
an entire relation. Next, if you like, and at run time (i.e. by code),
you can get rid of the last line, substituting it with a part of the
where clause as, e.g.,

select o.order_no, o.order_date, c.cust_name
from orders o, customers c
where o.cust_no = c.cust_no
and o.order_date >= :min_order_date

assuming that the aforesaid query returns a non empty recordset.

However, a no-fetch query like the former, is perfectly valid if used at
design time. And it opens very fast.

HTH

Giuliano
pcerdaz
2007-02-16 18:12:37 UTC
Permalink
Giuliano,

Your comments are being very helpful to me, however, the new
query with joined tables is not showing the field values of
new records that were inserted and populated, this happen
until exit and start again of my application.

To avoid this problem I’m trying with Commit of my database
and making a Refresh of the table, but it doesn’t work. Any
idea would be estimated.

Thank you,
pcerdaz
Post by unknown
Post by pcerdaz
At desing time I use the SQL Editor that create automatically somthing like "select * from MI_TABLE"
but at runtime I need some filters like "select * from MI_TABLE where A = B". Thinking again,
probably it is posible to add the joined fields no matter the filters that I applied later.
Exactly, you can place a "joined" query as well.
select o.order_no, o.order_date, c.cust_name
from orders o, customers c
where o.cust_no = c.cust_no
and 1 = 0
The line that contains "and 1 = 0" forms a no-fetch query, as the
expression 1 = 0 is always false (contraddiction or antitautology.)
A such query is used to permit to the DataSet component to retrieve
only the field definizions without loading the network to fetch
an entire relation. Next, if you like, and at run time (i.e. by code),
you can get rid of the last line, substituting it with a part of the
where clause as, e.g.,
select o.order_no, o.order_date, c.cust_name
from orders o, customers c
where o.cust_no = c.cust_no
and o.order_date >= :min_order_date
assuming that the aforesaid query returns a non empty recordset.
However, a no-fetch query like the former, is perfectly valid if used at
design time. And it opens very fast.
HTH
Giuliano
unknown
2007-02-16 21:03:09 UTC
Permalink
Il 16 Feb 2007 10:12:37 -0800, "pcerdaz" <***@manquehue.net> ha scritto:

Hi pcerdaz,

As far as I know, if you have modified the joined table in the context
of the same transaction object, you should be able to see the changes to the
recorset without commit the current transaction.

Hence, the obvious question is: have you refreshed the recordset?
That is, do you have closed then reopened again the DataSet component?
Keep in mind that the TIBDataSet's Refresh method, if I recall correctly,
is used primarily in order to refresh a single row, not the entire recordset.
In fact, you have to furnish a "refresh query" which includes a clause that
refetch the selected row in the opened recorset.
If you insert a record into a table using the TIBDataSet's Insert method,
fill the fields with sensible values, then you posts the new row using the
TIBDataSet's Post method, you can see immediately the changes you made.
But if you have added the new row with an another component (e.g. with
a TIBSQL componente), even if you are using the same TIBTransaction component
and you are in the context of the same transaction, alas, you can't see the
new row, e.g. in a grid, until you refresh the TIBDataSet component
connected with the aforesaid grid.
You could need to call the Commit method (or the CommitRetaining method) of
the Transaction Component, only if the row was added from another application
or, in the same application, if the row was added by others components linked
with a different transaction component (or in the context of an another
transaction).

Usually, it suffice to call the TIBDataSet::Close() and TIBDataSet::Open()
methods in sequence. That is:

IBDataSet->Close();
IBDataSet->Open();

However, it's possible that you lose the current cursor position. In order to
keep the current cursor position, you can use bookmarks. The following snippet
could help:

TBookmark const Bookmark = IBDataSet1->GetBookmark();
IBDataSet->Close();
IBDataSet->Open();
IBDataSet->GotoBookmark( Bookmark );
IBDataSet->FreeBookmark( Bookmark );

But, the aforesaid code isn't too elegant. Surely is not "exception safe"
code. Using the RAAI idiom is possible to have a better code. Just declare a
class like the following:

template<typename DS>
class BookmarkManagerType {
public:
explicit BookmarkManagerType( DS& DataSet )
: ds_( DataSet ), bm_( ds_.GetBookmark() ) {}
~BookmarkManagerType() throw() {
try {
ds_.GotoBookmark( bm_ ); // can throw?
}
catch ( Exception& E ) {}
ds_.FreeBookmark( bm_ );
}
private:
BookmarkManagerType( BookmarkManagerType const & );
BookmarkManagerType& operator=( BookmarkManagerType const & );

DS& ds_;
TBookmark bm_;
};

and use it in this manner:

void RefreshDataSet( TIBDataSet& DataSet )
{
BookmarkManagerType<TIBDataSet> BookmarkManager( DataSet );
DataSet.Close();
DataSet.Open();
}

Just call

RefreshDataSet( *IBDataSet );

in order to refresh the dataset IBDataSet without losing the cursor position.

Uhm, maybe we could try to apply again the RAII idiom even for the
open/close sequence. We add the following class:

template<typename DS>
class DataSetRefreshManagerType {
public:
explicit DataSetRefreshManagerType( DS& DataSet )
: ds_( DataSet ) { ds_.Close(); }
~DataSetRefreshManagerType() throw() { ds_.Open(); }
private:
DataSetRefreshManagerType( DataSetRefreshManagerType const & );
DataSetRefreshManagerType& operator=( DataSetRefreshManagerType const & );

DS& ds_;
};

and we transform the RefreshDataSet function in this fashion:

void RefreshDataSet( TIBDataSet& DataSet )
{
BookmarkManagerType<TIBDataSet> BookmarkManager( DataSet );
DataSetRefreshManagerType<TIBDataSet> DataSetRefreshManager( DataSet );
}

Good luck.

Giuliano
Post by pcerdaz
Giuliano,
Your comments are being very helpful to me, however, the new
query with joined tables is not showing the field values of
new records that were inserted and populated, this happen
until exit and start again of my application.
To avoid this problem I’m trying with Commit of my database
and making a Refresh of the table, but it doesn’t work. Any
idea would be estimated.
Thank you,
pcerdaz
Loading...