Hi just using tthis software for the first time
I sent 131 emails to the queue (as shown in overview) but only 27 actually got sent. the others remain in the queue bit are not being sent (as per log)
pretty much default settings – eg queue interval 5 minutes
what is wrong with my settings????
May 21, 2020 at 6:39 am #63624
and how do i release them from the queue so they actually get sent
May 21, 2020 at 9:57 am #63627
If you mean the content of php_errolog then there is nothing in it since 1 entry may last year.
I also assume php error logging is turned off by default so I won’t be able to see any error after the eventwould I?
What can you tell me about your cron job. i suspect it has stopped *because of some error as you say).
but the overview says it continues every 5 minutes
but i can’t see any cron jobs defineed and running on my site cpanel
May 21, 2020 at 10:39 am #63631
The problem is that if the error happens, emails that were in the processing status get locked, and ignored with the next run. There will be some updates to this in the next update, including the reset tool, but I have no ETA for the next update yet (hopefully early next month). You can use phpMyAdmin or some tool like that and in DB table ‘wp_gdmaq_queue’ change all the rows that have column ‘status’ with value ‘waiting’, and change that ‘waiting’ to ‘queue’. And, the next time queue runs, it will pick up these emails.
WordPress has its own pseudo corn job running, so it doesn’t depend on the server. And, you should always have logging of errors enabled on the server level, because errors in WordPress and plugins happen all the time, and it is very useful to be able to find these issues.
May 21, 2020 at 10:56 am #63633
looking forward to the upgrade. – asap for me would be real good.;)
doesn’t the pseudo cron job relay on web site activity some how? My site is pretty quite but i still need all emails to go out if some one post a topic in my forum. or is that enough site activity to keep the pseudo cron job ticking over.
I’m running this queue on my production site. If I set debug= true in my wpconfig.php file wont errors be displayed for my clients to see. or is there a way for me to just log them only in the log file
May 21, 2020 at 11:43 am #63634
OK, I have checked the development plans, and the GD Mail Queue Pro 4.4 will be released on June 1, 2020 with Lite 3.5 coming out day or two after that. Both will include automatic requeue for the emails waiting.
If your website is not very active, pseudo cron will be running with delay, that is for sure. You can replace pseudo cron with server cron, there are many guides for different hosting companies, here is the official WordPress information: https://developer.wordpress.org/plugins/cron/hooking-wp-cron-into-the-system-task-scheduler/.
You can configure WordPress to log errors: https://support.dev4press.com/kb/article/enable-wordpress-debug-mode/ and it can be set to log and not display.
May 21, 2020 at 11:46 am #63635
ok thanks for the links I’ll followup
June 1 is great and not far away.
looking forward to it.
May 28, 2020 at 6:26 am #63700
Just did another test today and now I need help deciphering the “queue cron Job” parameters settings.
my current settings are
Interval = 15 minutes
max job length = 20 secs
emails per job = 150
flexible job = enabled
change from = enabled and new values set
change email reply = not enabled
change sender email = enabled and value set
add process header = enabled
phpinfo says I have Max_execution_time = 120
so does that mean I can set max job length to 110 secs without ramifications
I don’t have any restrictions on the number of emails I can send at once (to my knowledge anyway) as I am using a pay for use smtp provider – so sending 154 emails at once should not be any problem.
1) I only got 26 emails delivered (of 154) the first time the queue ran – yet I set it to 150 I thought. Did the queue stop after 20 secs and in that time my site could only process the sending of 26 emails to my smtp provider?
once this queue run stopped there were 4 emails ‘left in the queue’ (as per overview and as I expected there would be) but 154-26 left in log “queue” – this we have discussed above so I understand. (There are no entries in the debug error log showing why the queue stopped – as you asked about in the above).
the next time the queue ran, the 4 remaining in the queue (as per overview) were delivered like the previous 26 – as I expected they would.
2) How do I get more emails to be processed each time the queue runs? Is the limiting factor in my case (since I no number of email sending limits) the max job length (ie max_execution_time). So setting max job length to 110 would mean that I would get about 5*26 emails processed each queue run?
3) What is the purpose of “flexible job” setting and how does it work out that “the plugin will attempt to send one more batch of emails during the same CRON execution”? I assumed since I had it enabled and I have a max job length of 20 which is way less than max_execution_time then this should have setup another 4 or 5 attempts to send another “batch of emails” as I had told it to process 150 “emails per job”. How big is a “batch of emails”?
May 28, 2020 at 10:28 pm #63710
You are mostly right about this! The first thing plugin does is to take 150 emails and run it, no matter how long timeout is. Timeout is checked after the first batch is sent. But, in your case, batch of 150 starts processing, and only 26 emails get delivered. That means that during the server timeout period of 120 seconds, only 26 emails got delivered. Many servers don’t log errors caused by the server timeout, and they silently fail. This appears to be happening to you here.
As for the flexible job, if you set it to 5 emails and 50 seconds, plugin takes 5 emails, run it, check the time, and if there is enough time left, it runs another cycle, until it reaches timeout.
If your server really has 120 seconds timeout for PHP and if in that time only 26 emails got delivered, that is not very slow. I am not sure about the delivery method used (mail() or SMTP), but for both, this is very slow. And, also, not surprising. Many shared hosting have a email delivery limits that the hosting company doesn’t disclose, and it might be that your hosting has very conservative limits set.
I use the SiteGround VPS server, and in 30 seconds timeout period, I can send more than 800 or 1000 emails without issues. But, in the past, I was using BlueHost and Hostgator, and in some cases, it was sending under 50 emails in 30 seconds timeout.
May 29, 2020 at 11:14 am #63713
I too use SiteGround but a shared server in Singapore – I don’t have the skill set to manage a VPS.
We have been sending up to 45 emails regularly using their web mail client and BCC’s. I think they also have a max number per a rolling 15min window as well for these sorts of emails.
I have previously sent about 50 emails (of 100+) from my web site using just wpmail and had to abandon this because the rest got lost and there was no consistency in which 50 of the 100 would actually be sent. But now I know of GD mail Queue I’m trying again – and using a paid for use smtp service which I thought would overcome the shared server email limits.
I’m not sure I’ve told you about how I generate these emails. Apart from WordPress standard transactional (including Wp_members versions of these), I am using Asgaros forum plugin on my site and am experimenting with his notification emails for forum and topic subscribers. I currently have 130 members in my forum (members only) and they are all subscribed to one particular forum by default (until they chose to unsubscribe anyway). Hence my testing using batch of 154 emails (22 email ‘new topic’ notices sent to 7 of my personal email addresses) from my test WP site – as close as I can get to emulating my 130 members on my production site.
So after a long and not very productive support chat with SiteGround I have done another test today
It seems I am going to have to accept 26 emails per 120 seconds as my SiteGround account has a fixed max_execution_time of 120 sec. The only way around this is to upgrade my account (to something like yours I think which apparently don’t have this 120 sec limit)
I’m waiting response from my smtp service to see if there is any possibility of a lag due to internet connectivity between SiteGround and their servers – not that I expect there will be anything I can do in the short term if there is.
So I changed my settings to
Interval = 5 minutes
max job length = 50 secs
emails per job = 7
flexible job = enabled
then ran a test of 70 emails in the queue
First scheduled run of gd queue sent 21 to smpt server and were delivered. So flexible job = enabled is working I think, 3 times.
49 emails remained in the queue
then the next scheduled cron job ran and
21 more emails were sent
and 28 remained in the queue
then the next scheduled cron job ran and
21 more emails were sent
and 7 remained in the queue
then the next scheduled cron job ran and
7 more emails were sent
and 0 remained in the queue
What would likely to happen if I set max job length = 100 secs? Would it theoretically double the number of times the queue would run because flexible job is enabled? Is 100 ‘too close’ to my 120 max limit?
So a major milestone – all emails actually processed and nearly all sent and received.
well nearly because – the first 2 emails of the last run (of 7) failed to send but the next/last 5 were sent. The 2 email recipients involved had previously received emails as part of each of the previous 21 emails batch.
I can’t find any logging of the reason why the send failed.
It seems when a send fails there is no reporting of it, eg a bounce back to the sender. Am I missing something or is this a Pro feature or something you can’t determine and then report.
I could actually live with this delivery rate (20-25 minutes for 130 or so emails – for now anyway) for my forum notifications given my low traffic volume site. But I can’t wait 30 minutes for things like password reset emails. So being able to exclude these types of emails from the queue becomes important. So I would appreciate it if you would check out the asgaros forum plugin (as you indicated you would for wp-members) and see if you can tell if an email is originating from within it or not. Then a suggestion – perhaps your plugin could offer ‘include emails from…..’ rather than the current ‘exclude emails from ….’ logic. If this could be made available I would be able to say ‘include all from asgraos forum but exclude everything else’.
May 29, 2020 at 2:07 pm #63714
well, if you are sure about the 120 seconds timeout, I would advise using smaller queue size (10 emails), enable flexible sending, and set plugin timeout limit to 90 seconds to be sure (the last batch can start on second 89, and it can take time). This will ensure that everything will be sent. As for the errors reported on sending, my plugin has no control over the process after it sends the email, and it will get only basic error information that usually is not very helpful. Also, some deliveries may be sorted as spam or something else, so I can recommend having both senders and reply to set for your emails.
As for the delivery speed, when you use only WP delivery is faster, because email is plain text. If you use HTMLfy with my plugin, the resulting email is larger, so it takes longer to send. While SiteGround shared hosting is really good, all shared hosting have bad email servers with a lot of limits that they can’t handle differently for the price point, and that is why VPS is always better solution when dealing with large email traffic.
As for exclude/include, the filter used for that I explained previously can be used for both cases, include or exclude by the email type, so you can write custom code to exclude all emails from capture, except those with selected types. There is no way for me to create absolutely precise interface to control all that, because detection process is flawed, because WordPress default mail system is bad to begin with (that is why plugins like this one are needed, but limited with WordPress limitation).
May 30, 2020 at 4:00 am #63719
Thanks for your comments on settings values
Yes I have to use HTMlfy as my smtp service requires html for transactional emails.
I’ll have to do much researching on using the filter as I am not a php coder.
I’ll wait for the up coming new release and see how it goes.
I really appreciate the work you have undertaken to write this app. I remain amazed the WP email system hasn’t kept up with modern technology given its blogging history.
June 10, 2020 at 3:33 am #63834
Well thank you that is great
BUT might have a problem or two
All on my dev site so not affecting my production site yet.
I am now re-receiving emails from before – ie emails I got from previous testing. Not sure just how many just yet but more than 100. I saw 94 emails in the queue according to the overview. there have also been a number of ‘locked in queue’ and this number keeps changing each time the queue runs
I have also installed ‘query monitor’ plugin and it gave me an error during updating process – sorry have lost the exact message but it was a missing file from a download folder (on wordpres.org i think) and I think the file had a “.sig” extension – at least that is the last bit i could see in the error message. Apart from that the update was successful.
June 10, 2020 at 4:08 am #63835
Well my smtp service said it sent 124 emails for me (today so far) and they all appear to be from a previous test run when I tried to send 154. But it seems to have not resent the first 30 or so of that test. I think it looks like the first queue execution (of about 25 emails – yes 25 vs 30 don’t match up but are close) were sent and flagged in the db as being sent but the remaining 124? which were still in the queue but ‘locked’ because the queue run aborted because of php max_exectution_time_limit crash. And now the automatic resetting of ‘locked entries’ in the queue has finally sent them.
I had deleted these ‘locked’ entries from the log thinking that would delete them from the database of emails waiting to be sent – but maybe that didn’t happen..
If so, I think I will have the same problem on my production site as that where I first tried to send my ‘bulk’ email test and only 25 or so of my intended 132 were actually sent. If so I am happy to manually delete database entries using phpadmin but i just need a bit of guidance to make sure I would be deleting the correct entries
June 10, 2020 at 9:45 am #63836
The queue has own database table: ‘wp_gdmaq_queue’, emails that are sent by the Queue are stored there. Emails with ‘waiting’ ‘status’ are those that were locked when the PHP timeout killed the execution, and new plugin version will change their status back to ‘queue’. And, you have ‘sent’ and ‘fail’ too.
June 10, 2020 at 9:51 am #63837
so if i don’t want to send the ones that are now ‘waiting’ then I have to clear the database. correct?
I have found a way to clear databases (and statistics) using the tools menu option. I guess i use the clear databases on my production site before i upgrade so as NOT to send the unwanted emails. correct?
I have used these 2 tools on my dev site and now am unable to set the queue interval. it defaults to 1 minute and no matter what i put into the field or use the updown arrows to select a value when i click save settings the screen refreshes and the field is blank again. and the queue interval defaults to 1 again,
Fairly sure it was set to 5 minutes before i used both tools – it was before the upgrade anyway and i could set it then as well
June 10, 2020 at 10:39 am #63841
Minor 3.5.1 has been released to fix the issue with Cron value saving.
As for the database, you can always clean the ‘wp_gdmaq_queue’ table. This table is auto clean every day to remove old sent/failed records anyway (older than 30 days).
June 10, 2020 at 10:43 am #63842
ok – but its not been 30 days and i have 104 ‘waiting’ in the database so i’ll need to manually clean it then
thanks for the big fix – applied and it works
- You must be logged in to reply to this topic.