This topic is locked

Searching on aggregate values

10/20/2006 6:42:32 PM
PHPRunner General questions
T
thesofa author

I need to search a set of records using advanced search where the search field is an aggregate field, i.e. it is in the SQL query as select COUNT(ID) AS wibble,
when I try to search this value, i get an arror which points me to the fact that the searching is done in an order that prevents this happening

The WHERE clause may not be used for this. The WHERE conditions are considered before the aggregation.



It seems that I need to use the Having word to filter or search,

Where do I alter the code to suit this search?

Cheers

Admin 10/23/2006

Hi,
I recommend you to create a view in your database with CREATE VIEW command then use this view in PHPRunner.

Then you'll be able to search on aggregate values without any code modifications.

T
thesofa author 10/23/2006

Hi,

I recommend you to create a view in your database with CREATE VIEW command then use this view in PHPRunner.

Then you'll be able to search on aggregate values without any code modifications.



My fault this, I was not explicit enough.

The report is already based upon a view made in PHPRunner, will making a view in the MYSQL databse, V5, make any difference?

I need to be able to do an advanced search on three different fields, so if I create a view in the MYSQL database, rather than through PHP, will I be able to search `TutorGroup`CONTAINS "xxxx" AND `DateGiven` BETWEEN xx/xx/xxxx and xx/xx/xxxx AND Count(`detainee`)>=3?????
The full sql query is as follows

select `ID` ,

count(ID) as Total,

`DateGiven`,

`detainee` ,

`donor` ,

`TutorGroup`,

`DayForDetention`,

`sess`,

`Lunchtime`,

`Done`,

`Origin`,

`MoveToDept`,

`ReasonForDept`,

`pupils`.`lname`,

`detentions`.`OriginalDate`,

`reasons`.`reason` AS `Reason Given`,

`detentions`.`DoneAtDept`,

`sessions`.`sess_time`,

`detentions`.`MovedToAfterSchool`,

concat(`pupils`.`fname`,' ',`pupils`.`lname`) AS `Pupil`,

`detentions`.`reason`,

concat(left(`staff`.`firstname`,1),'. ',`staff`.`lastname`) AS `Teacher`,

`dept`.`department`,

`detentions`.`DeptHead`

from ((((`detentions`

inner join `pupils` on((`pupils`.`idPupil` = `detentions`.`detainee`)))

inner join `reasons` on((`reasons`.`idReason` = `detentions`.`reason`)))

inner join `sessions` on((`sessions`.`sess_id` = `detentions`.`sess`)))

inner join `staff` on((`staff`.`userid` = `detentions`.`donor`)))

inner join `dept` on((`dept`.`idDept` = `detentions`.`department`))

Group By

`detainee`



Cheers and thanks for the help

Admin 10/23/2006

Hi,
create a view by issuing the command in MySQL:

create view myview as

select `ID` ,

count(ID) as Total,

`DateGiven`,

`detainee` ,

`donor` ,

...


Then you'll be able to search for this condition:

`TutorGroup`CONTAINS "xxxx" AND `DateGiven` BETWEEN xx/xx/xxxx and xx/xx/xxxx AND Total>=3

T
thesofa author 10/23/2006

Once again, I am not being clear in my request, i feel.

I need it to filter out those detentions given between 2 dates, group them by detainee, THEN it has to count the detentions given per detainee, Then it has to filter out only those with 3 or more.

The view you gave me does not give the correct results when I do that. It shows the count of all the detentions given to that detainee, not just those given during the specified period.

I then need to be able to filter the count field.

its like a query in a query, but I need a filter on the inner query as well as a count on that filter, then a filter on the outer query.
I am stuck.

I can do it on my thumbs, using php makes this harder.

Admin 10/23/2006

Hi,
I see what you saying.

You can replace WHERE with HAVING in generated ..._list.php file.

Find this line there:

$strSQL = AddWhere($strSQL,$strWhere);



and replace it with:

$strSQL .= " having ".$strWhere;



Then find this one:

$strSQL = AddWhere($strSQL,$sWhere);



and replace it with:

$strSQL .= " having ".$sWhere;


I don't guarantee this would work with Master-detail relationship or with Advanced Security.

T
thesofa author 10/23/2006

Hi,

I see what you saying.

You can replace WHERE with HAVING in generated ..._list.php file.

Find this line there:
and replace it with:
Then find this one:
and replace it with:
I don't guarantee this would work with Master-detail relationship or with Advanced Security.



Aha, a sneaky one that, I have been digging in the xxx_search.php file, perhaps that is why I found nothing <img src='https://asprunner.com/forums/file.php?topicimage=1&fieldname=reply&id=12057&image=1&table=forumreplies' class='bbcemoticon' alt='<<' />

T
thesofa author 10/23/2006

I have tried this and it produces a query like this

select count(ID) as Total, `DateGiven`, `detainee` , `DayForDetention`, `sess`, `Done`, `Origin`, `pupils`.`lname`, `sessions`.`sess_time`, concat(`pupils`.`fname`,' ',`pupils`.`lname`) AS `Pupil`, `detentions`.`reason`, concat(left(`staff`.`firstname`,1),'. ',`staff`.`lastname`) AS `Teacher` from ((((`detentions` inner join `pupils` on((`pupils`.`idPupil` = `detentions`.`detainee`))) inner join `reasons` on((`reasons`.`idReason` = `detentions`.`reason`))) inner join `sessions` on((`sessions`.`sess_id` = `detentions`.`sess`))) inner join `staff` on((`staff`.`userid` = `detentions`.`donor`))) inner join `dept` on((`dept`.`idDept` = `detentions`.`department`)) Group By `ID` having 1=1 and `detentions`.`DateGiven`>='2006-10-02 00:00:00' and `detentions`.`DateGiven`<='2006-10-13 00:00:00' and `detentions`.`Total`>=3 ORDER BY `pupils`.`lname` ASC


and an error

Unknown column 'detentions.Total' in 'having clause'


so I played around with the query and got this to work

SELECT

Count(`detentions`.`ID`) AS `Total`,

`detentions`.`DateGiven`,

`detentions`.`detainee`,

`detentions`.`DayForDetention`,

`detentions`.`sess`,

`pupils`.`lname`,

concat(`pupils`.`fname`,' ',`pupils`.`lname`) AS `Pupil`,

`detentions`.`TutorGroup`

FROM

((((`detentions`

Inner Join `pupils` ON ((`pupils`.`idPupil` = `detentions`.`detainee`))))

Inner Join `sessions` ON ((`sessions`.`sess_id` = `detentions`.`sess`))))

WHERE `detentions`.`DateGiven` >= '2006-10-02 00:00:00' AND

`detentions`.`DateGiven` <= '2006-10-13 00:00:00' AND

`detentions`.`TutorGroup` LIKE '7%'

GROUP BY `detentions`.`detainee`

Having Count(`detentions`.`ID`)>=3

ORDER BY `pupils`.`lname` ASC


This produces the correct answer from the tables, having filtered them by hand in Navicat, the numbers check out.
If I try this as a filter

SELECT

Count(`detentions`.`ID`) AS `Total`,

`detentions`.`DateGiven`,

`detentions`.`detainee`,

`detentions`.`DayForDetention`,

`detentions`.`sess`,

`pupils`.`lname`,

concat(`pupils`.`fname`,' ',`pupils`.`lname`) AS `Pupil`,

`detentions`.`TutorGroup`

FROM

((((`detentions`

Inner Join `pupils` ON ((`pupils`.`idPupil` = `detentions`.`detainee`))))

Inner Join `sessions` ON ((`sessions`.`sess_id` = `detentions`.`sess`))))

GROUP BY `detentions`.`detainee`

Having `detentions`.`DateGiven` >= '2006-10-02 00:00:00' AND

`detentions`.`DateGiven` <= '2006-10-13 00:00:00' AND

`detentions`.`TutorGroup` LIKE '7%' AND Count(`detentions`.`ID`)>=3

ORDER BY `detainee` ASC

it only returns half the records.. It would seem that the use of having excludes any pupils who have detentions outside the date range specified, even if they have some inside the range.
I believe I need to have a query string based upon the date and tutor group in a where clause with the total count being in the Having part.

I am quite happy to Hack the final code to produce this effect, only I will need guidance as to where and having, Where do they go in the code?
I am now almost totally bald, but I should like to crack this one, with all of your help please.
You told me to replace various strings with new ones, is the first one for simple search and the second for advanced search?
Is there any way I can use the totalling/count feature on step 8, field order and totals to count the detainees records?

T
thesofa author 10/24/2006

bump

Admin 10/24/2006

Hi,
the first replacement I gave you is for simple search, the second - for advanced.
I'm not sure what results do you expect from your SQL query.

Possibly you need to use both WHERE and HAVING clauses. I.e. something like this:

select

...

WHERE

`detentions`.`DateGiven` >= '2006-10-02 00:00:00' AND

`detentions`.`DateGiven` <= '2006-10-13 00:00:00' AND

`detentions`.`TutorGroup` LIKE '7%'

GROUP BY `detentions`.`detainee`

Having Count(`detentions`.`ID`)>=3

ORDER BY `detainee` ASC


PHPRunner doesn't deal with HAVING clause at all.

You can modify generated ..._list.php file to construct complex search queries.

Here is the snippet int he file that constructs Advanced Search SQL string.

else if(@$_SESSION[$strTableName."_search"]==2)

// advanced search

{

$sWhere="";

foreach(@$_SESSION[$strTableName."_asearchfor"] as $f => $sfor)

{

$strSearchFor=trim($sfor);

$strSearchFor2="";

$type=@$_SESSION[$strTableName."_asearchfortype"][$f];

if(array_key_exists($f,@$_SESSION[$strTableName."_asearchfor2"]))

$strSearchFor2=trim(@$_SESSION[$strTableName."_asearchfor2"][$f]);

if($strSearchFor!="" || true)

{

if (!$sWhere)

{

if($_SESSION[$strTableName."_asearchtype"]=="and")

$sWhere="1=1";

else

$sWhere="1=0";

}

$strSearchOption=trim($_SESSION[$strTableName."_asearchopt"][$f]);

if($where=StrWhereAdv($f, $strSearchFor, $strSearchOption, $strSearchFor2,$type))

{

if($_SESSION[$strTableName."_asearchnot"][$f])

$where="not (".$where.")";

if($_SESSION[$strTableName."_asearchtype"]=="and")

$sWhere .= " and ".$where;

else

$sWhere .= " or ".$where;

}

}

}

$strSQL = AddWhere($strSQL,$sWhere);

}


Modify it to add both Having and Where clauses to $strSQL.

T
thesofa author 10/24/2006

Hi Alexy, thanks for the snippet, I had come to the same conclusion as you, I need WHERE and HAVING.

I am more than happy to hack code, what I need to know is the variable names for the search criteria from the advanced search page, so if I have the following fields on the advanced search page, in this order

TutorGroup

DateGiven

DayForDetention

Total......Count(`detentions`.`detainee`)
What will the variables be, and will it vary if I only use 3 of them rather than all 4?If I use TutorGroup, Dategiven and Total, will that change the variable names used to build the sql query?

Many thanks for all your help, I seem to be getting more than my fair share, but I am trying to put some back too!

Cheers

T
thesofa author 10/25/2006

bump

Admin 10/26/2006

Hi,
$f variable in the code contains a field name.

$where - a search criteria for a given field
all you need is to combine search criterias properly.

T
thesofa author 10/26/2006

Hi,

$f variable in the code contains a field name.

$where - a search criteria for a given field
all you need is to combine search criterias properly.



Hi

OK I admit it, I have no idea where to start, please reduce status to Noob.

what I would like to do is get the search string and just before it gets fired into the mYSQL server, add a clause at the end along the lines of "HAVING Count(`detentions`.`detainee`)>=3"

so that the query only displays the records where they have 3 detentions in the period set by the advanced search options.

If the staff want to change the number of detentions that triggers another letter home, then I will change it in the code by hand.

Please help me ONCE AGAIN.

Cheers

T
thesofa author 10/29/2006

I have been thinking for a change, if I create a temporary table of the data I need for the search to work, I can have a column of the count of the detentions, with the value created as the table is written. This table will have to be named with the name or identifyer of the logged in user, and it will have to be emptied and dropped after each use.

I can hear the sound of thrashing hard discs here. This will only need to be used when the user tries to filter on the Total column, so I will need some decision making and branching here.

Please can you tell me whereabouts in the code created by PHPrunner is the finalised sql search string creted before it is used to query the database?

Then I can try to create or re-work the query to suit my needs?

Cheers

T
thesofa author 10/29/2006

I think I have cracked it.

I have a field in the sql query of

Count(`ID`) AS `Total`,



As I always want a count of detentions, but between certain variable date and variable year groups, But I always want the details from all those who have 3 or more detentions in the given period.

So I have to modify the search parts of the xxx_list.php files.

find the part around line 248 (in mine anyhow)

that reads

// order by

$strSQL.=" ".trim($strOrderBy);



above this is the line like this

$strSQL = AddWhere($strSQL,$sWhere);



add this snippet to the end

." HAVING `Total` >= '3'"



so the line reads

$strSQL = AddWhere($strSQL,$sWhere)." HAVING `Total` >= '3'";



this adds the 'HAVING' clause just before the 'ORDER BY' statement and produces correct results.

OK so I have to add the code in each time I rebuild, but it works.

Hack, Hack ,Hack....