The Event – Technical Analysis of Downtime Due to Lack of Audio Data via WebRTC RTP on August 31st, 2020.

General Timeline:

  • Sunday ~3AM -6AM PST – Level 3 / CenturyLink goes down. Cloudflare is affected taking down major services and websites globally. All traffic to Level 3 returns 500 error codes (Missing or Invalid Routes).
  • Sunday ~6AM PST – Cloudflare reroutes traffic around CenturyLink, some services are still affected particularly those on or near CenturyLink endpoints.
  • Sunday ~9 AM PST – Level 3 / CenturyLink comes back online.
  • Monday ~6:00 AM PST we started receiving reports of lack of audio data to the Dialer.
  • Monday ~10 AM PST we noticed that is was only effecting certain accounts, making it more difficult to troubleshoot and isolate. We begin a session with Amazon to investigate any network troubles that could be causing this.
  • Monday at 1:05 PM PST – We identified the issue and began patching Dialer servers. The issue was that Google’s STUN Servers were not identifying our domain names correctly causing bogus connections when connecting to dialers via WebRTC. This was identified to have been a badly cached DNS route by Google due to the Sunday disturbance at Level 3. We switched to our Telco Provider’ss STUN Servers to mitigate further issues and also to have access to logs and service our Telco can provide that Google will not.
  • Monday ~1:45PM PST – Last of the fixes concluded around 1:45PM and everyone resumed operations.

Read about CenturyLink / Level 3 Outage Post Mortem by Cloudflare

Steps to Monitor and Prevent from Occurring Again

  • Only use Telnyx STUN and TURN Servers for WebRTC Signaling instead of relying on Google’s Public STUN Service.
    • This provides a reliable source of STUN and TURN specifically for our purposes.
    • We have a great relationship and quick response from Telnyx, so if any issues or changes occur with their service we are notified and we can investigate logs and traffic data.
  • Removed Google and alternative STUN and TURN Servers from the Client Side Webphone.
  • Switch to IBM Quad9 DNS ( internally instead of Cloudflare ( due to the recent upsets at Cloudflare and their upstream providers.
    • If any similar issues arise investigate Dialer DNS’s settings first.
  • Change TTL for all Dialer Subdomains back to 300 to prevent Global Outage DNS Cache issues like this from effecting us for longer than 5 minutes.

The Beginning

We were tearing our hair out at the beginning of this fiasco because nothing had changed over the weekend. No development, no updates, and Friday all was normal. Once we started getting reports, we started scouring logs, checking connections, restarted servers, recommissioning servers to change the hardware zones, and nothing was working. All systems were nominal. HTTP Traffic was perfect, load was perfect, web sockets were connecting without a problem, the only issue was that audio data was lagging out, not connecting, not being received and generally not working.

Suddenly, Exceptions

With over 10 Dialers now inoperable, things were beginning to escalate but upon checking all accounts we realized that a few were still working fine. We then assumed that it was a networking issue at Amazon with the specific instances, albeit strange that they should all happen at the same time. We have never such a massive failure on multiple Dialer’s before, because we have many subsystems to separate concerns and keep the system going should one fail. But without audio, it is hard to run a call center. There was also no indication as to errors or problems occurring in any logs on any of the machines. We then contacted Amazon AWS and spent a few hours with their networking technicians and support to no avail. RTP Traffic WAS flowing, it WAS being accepted, it JUST wasn’t working on the Webphone.

In the case of a single server having this issue we would have taken action to install Softphones to circumvent the issue temporarily, but deploying Softphones across offices, during COVID with many working from home, would have taken just as much if not more time than to continue to investigate the issue so we made the decision to keep investigating. Also, Softphones don’t play well with some of our cluster setups that require multiple registrations for load balancing, so this was not an ideal solution.

An Epiphany: It’s Not DNS, There’s No Way it’s DNS, It was DNS, but with a Twist due to the Sunday Event…

We checked DNS as one of our first checklists as to issues that could be, we updated servers with different DNS settings, moving from Cloudflare DNS Servers to IBM, thinking maybe the servers were being effected by the issues at CloudFlare and were broadcasting the wrong IP address, had a bad cache or something strange. This was not the case and didn’t help much, but suddenly however, one of the servers started working while we were testing this theory. We had no idea why. 30 Minutes later another one started working, Audio just started flowing. I got to thinking about the Sunday Event and how this could be a ripple effect of the Event.

I was up at 3AM Sunday enjoying a leisurely video game when everything came crashing down, cannot login. Multiple Major Game Services and other services all around the country and world were suddenly inaccessible. Most of these services use CloudFlare, and one of CloudFlate’s Upstream providers had just thoroughly and globally crashed, causing requests to Cloudflare that are forwarded to websites to return as if they did not exist, or were problematic.

I thought…what if servers across the world did a DNS Lookup request during this event, and then got corrupted, it would then point to an invalid IP Address, or no Address, which would effect our subdomains individually. Subdomains have different refresh rates for DNS Cache, anywhere from 300 seconds to a full day. And we just so happened to have most of ours set to a full day due to DNS Lookup issues our overseas clients were sometimes having due to poor internet.

We then started looking at the internals of how the Webphone works, because we were able to get the softphone working without issue, so it must have to do with WebRTC, or a Protocol Client-side that is not negotiating properly between the client and server.

The fact that one of our servers came back meant that the DNS cache for that Dialer’s subdomain suddenly expired and refreshed on it’s own, so it now pointed at the valid IP.

But the question was WHERE?! What service had this invalid cache? Nothing was returning invalid or erroneous values in our server logs. Our Wireshark and SIP Tracing was literally just showing NO data being received by the client even though it was being sent! Everything else was connected and working! The website was accessible, pings worked, there seemed to be no DNS Cache issue as we were able to access via HTTP, SSH, and pure RTP without any issues!

The Answer was: Google’s Stun Servers

We found the answer inside the Webphone. The Webphone prefers our telephony companies STUN Server per our configuration, and then falls back to a global standard Google Operated STUN Server that ships as default with most telephony systems that work with WebRTC. Google itself had an invalid pointer to our servers, the bad cache was on their end where we couldn’t see it, likely due to the Sunday disturbance that returned error messages for so many websites across the internet when requested. We also found that the Dialer’s themselves, outside of the client, had Google set as the Default STUN Service, which ships as the standard default STUN service for most WebRTC applications. This was making Google the preferred STUN service and although our Carrier’s Telnyx, was sometimes checked, it was not always the case. This was the missing key.

The offending Server was:

What is STUN and TURN?

  • STUN Technical:
    • Session Traversal Utilities for NAT (STUN) is a standardized set of methods, including a network protocol, for traversal of network address translator (NAT) gateways in applications of real-time voice, video, messaging, and other interactive communications.
  • STUN English
    • STUN determines where to send audio data to your computer when you are on a network at home or at a coffee shop. Since you share an IP Address with everyone on your network, you need to be identified as to where you are to send the audio stream.
  • TURN Technical
    • Traversal Using Relays around NAT (TURN) is a protocol that assists in traversal of network address translators (NAT) or firewalls for multimedia applications. It may be used with the Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). It is most useful for clients on networks masqueraded by symmetric NAT devices.
  • TURN English
    • Similar to STUN, except when we can’t figure out where exactly you are on your home network due to incorrect or strict settings, TURN is used as a fallback to Relay the Audio information between yourself and a server.

The Results

We REMOVED Google’s Stun Server on the Dialer Backend, replacing it with Telnyx’s ( which we preferred anyways because we have access, communications and a good relationship with Telnyx to debug things quickly when issues arise ) and SUDDENLY audio is working.

We quickly deployed this to two other servers with success, and after a few minutes of manually configuring each Dialer we were back in action.

Likely, this issue would have cleared itself up in a few more hours due to DNS Cache refresh as the 24 hour mark approached for those subdomains configured with the 24 hour TTL (Time to Live). But we would have never known the root cause. Now we know.

As of now we have changed our TTL to 5 minutes for all domains and subdomains to prevent DNS Cache from being an issue in the future.

We have also removed Google and other alternate STUN and TURN Servers from our list of options on both our Webphone Client and Asterisk backend and are sticking with Telnyx.

As an extra level of protection and due to the notorious and concerning routing issues Cloudflare has been encountering the past few months ( See July 17th Cloudflare Outage caused by a single line of incorrect configuration code ) we also have opted to change our server’s DNS Settings to IBM’s Quad9 ( as we had already changed from Google’s DNS Long ago due to issues, ping and speed ).

Thanks and Mentions

I want to thank the wonderful staff at Amazon Web Services, particularly Joshua K., for staying on the phone with us for hours testing, analyzing, and for looking into the server specs and instances we have with them to find any outliers. Their customer service and cloud service was performing stellar as always.

I also want to thank the Good Folks and our Telephony Carrier at Telnyx of which we have been with nearly since their inception for being there for us and assisting with theories and ideas as to what this problem could have been. It is nice to know that they have our backs and we can reach out as friends and colleagues. Thank you Shreya, Rogelio and Zach for the assistance and conversation and for maintaining a stellar telephony product with amazing prices and awesome customer service.

Conclusion and Appreciation for our Clients

Lastly, thank you, our clients, for bearing with us during unforeseen event. It’s not every day that a backbone of the internet crashes so hard leaving ripples like this; I am glad that the Sunday Event didn’t happen Monday as it would have been even worse to track down, little less possible to correct.

We strive to make a better and stronger product in our niche and are always here for you guys.

Today, your revenue was lost, your agents are disappointed in us for not being able to make sales, and it was your payroll that was running on the clock as you waited.

Please know we worked our hardest to solve this issue and make the right decisions to get it fixed properly and as timely as we could. We considered stopgap solutions, but it would have taken longer to implement a stopgap than to get to the root cause and correct it.

2020 has been a rough year, and fortunately we have been able to keep up with the demand with people using our system nationally and internationally and we appreciate everyone of you, your feedback, and daily conversations in Slack.

We have optimized our services to be able to be used from anywhere, and added tons of features and functionality per your suggestions that have benefited all on the platform.

We will keep building and fighting for you and your business.

I was surprised and warmed with the calls of concern, and how understanding you all have been today; not a single angry email, Slack message or phone call. This makes me proud to be working with you all and in this industry.

I know it is not much, but we will be offering credits for this one day of downtime that will be directly applied to your current monthly invoices, I know it pales in comparison to the lost revenue of nearly a full day of productivity, but we want to do something to make things right and show our appreciation for you all being our clients.

– Alexander Conroy aka “Geilt the Architect”

Patch 7.5.5

Asterisk AMD Patch VICIDial Core Updates and TLDialer Replication Support

Asterisk 13.21.0 Patched and rebuilt with SIP Logging Capability and a fix to Asterisk 13’s AMD Hanging on No Audio Issue.
  • SIP Logging helps debug issues with calls that involves more than just Asterisk CLI, AGI, or AMI commands as it allows us to store and view this data in the database. It should only be on when needed.
  • The No Audio Patch addresses a bug in the AMD Application that caused it to hang in rare circumstances where it did not detect audio, and never close the channel until reboot. More noticeable with a higher volume of calls.

All Account VICIDial Instances have been updated from Version 2.14-714a Build: 190628-1511 to Version 2.14-735a Build: 200127-1620 as of tonight.

TLDialer Updates

  • RDS MariaDB 10.2.1 Read Replica Servers now implemented and possible upon request.
    • When Replication is Enabled, all Reports and TLDialer Sections revert to the replication server, so you can do crazy queries to your hearts content without fear of locking your operations up with ridiculous reports.
    • Replica updates are nearly real time, with just microseconds of delay.
    • Base Operations of TLDialer still function on the primary server.
    • Reporting in VICIDial itself can also use the Read Replica.
  • Fixed a bug with Custom DID Options for TLDSURVEY ( Automated Voice to Text and Text to Voice IVR + Survey System) and TLDBATMAN ( No Agent No Answer ) and TLDSRVDCL ( State Route Inbound ).
  • Fixed an issue with TLDSURVEY where the 9 Pause button was randomly being pressed.
  • Fixed Issue with Audio Checker where it would Require your computer to have audio devices even if you were using a softphone or hardphone instead of the TLDSIP Webphone.
  • Added Option to remove the “Qualification Time Countdown” from the TLDialer Interface. Currently this is a system setting only.

Vendors and Relays

  • Added Relay Delay
    • It is now possible to DELAY a Relay and instead QUEUE it for Relay at a specified interval. You can find the settings under Vendor Relay settings.
    • When a Relay is queued, all of it’s data is stored in the Relay Queue table, similar to how Relay campaigns work, except the date it can be released is set in the future. Every minute the relay queue is checked for relays that need to go out, so you can only delay by minutes for the queue system to pick it up.
  • Added “Inbound Price Override” setting to Vendors -> Inbound Tab.
    • This lets you override the leads cost if it is 0 or not the same as the current set price, if the lead exists. It also logs this in the leads action log, so it can only happen once, so feel free to clear the cost if there was an error. Also, if the QT Time (Qualification Timer) has cleared the cost, the next inbound will NOT update the price, as it was already modified by the system. This also can only happen with Inbound Calls (which is determined by the presence of a Number Dialed in Lead Pop / TLDialer Pop. This works with both External Systems that support Lead Pop and TLDialer.
  • Added Dupe Update IP Whitelist / Blacklist.
    • This allows you to receive leads from a vendor which will correctly reject dupes, but if you repost in the same leads from another system or dialer and want the dupes updated, but not rejected, then it will. This prevents you from getting erroneous successful posts from the Vendor when in fact…it was you all along causing your issues.

Commissions Paid Module

  • Commissions Paid is the reverse end of Commissions Received, and allows you to enter line items for who you paid and on what policy, along with breakdowns, even cash bonuses!
  • The Payroll Module, once complete, will enter data in Commissions Paid for calculations, so you are safe to enter data now as you see fit.
  • Date Paid Through is Optional, but can be helpful for the future and will be used with the Payroll Module.
  • You can select a User or a Vendor to tie to the payment. This way you can keep track of “Referrals” as “Vendors” if needed, so you don’t have to create false inactive visible Users to do the same job.
    • If you don’t have the User or Vendor to Set, you can just type in a name…but please try to keep some standards with the names if you expect to report on it.
    • The system will automatically calculate the “Recipient Name” based off your Choiced in the following Order.
      1. If User is Set, it will be First Name Last Name of the User.
      2. If Vendor is Set it will be the Description of the Vendor.
      3. If Payment Recipient is set it will fallback to whatever is typed in there.
  • Payout Types
    • Initial Payout
    • Residual Payout
    • Chargeback
    • Bonus Payout
    • Cash Bonus (Spiff)
    • Advance Payout
    • Other Payout
  • Payee People
    • Agent
    • Fronter
    • Vendor
    • Referral
    • Other
  • Amount Breakdown Categories – Must Equal “Amount“!
    • Base
    • Bonus
    • Cash (Spiff)
    • Chargeback
  • As with Commissions Received, we also have a Reference and Description for each record for your own internal used.
  • Coming Soon: Commissions Paid Data on the Contact Form.
  • Coming Soon: Auto Import of Current Commissions on Policies as Entries into Commissions Paid.
  • Coming Soon: Auto Calculation and Update of Policy Paid Data and Dates when using Commisions Paid, similar to how Commissions Received works.

Agency News Module

  • You can now create a template type under Media -> Templates called “News“.
  • You can show this “News”  article, along with who wrote it, and when it was last updated, on the Agent Dashboard.
  • A User Must be Part of a User Group that has a News Article set in the Users -> Groups Section.
  • An Agent can see Multiple News Articles if they are part of multiple groups with news.

TQL API Schema Updates, Core Updates and Fixes

  • Added Replication Server Compatibility. Allows us to Autoswitch between Live and Replication instances when enabled.
  • TQL Import now works with Comma Separated Import Names.
  • Fixed TQL Args Parser Order…somehow it still worked though?
  • Fixed TQL Aggregates to Coalesce to 0 for Sums.
  • Added User Group Members to Egress API.
  • Added Personal Phone to API User Meta.
  • Added a new type of Egress Endpoint for Crazy Cross Database Queries  as needed. They will be aptly named /custom1, /custom2, etc.
  • Added Date Owned Pseudo field, worked similar to Owner ID.

New Abilities

  • Added Unconvert Lead Ability.
  • Added View Verifications Ability.
  • Added View Submissions Ability.
  • Added View Clients Ability.
  • Added View Post Dates Ability.

Miscellaneous Updates and Fixes

  • Added Lead Status to Callbacks Table.
  • Added “Is Sale” and “Is Active Sale” to Policy Campaigns ( Triggers )
  • Removed Default’s of 0 for some Booleans on leads that should have actually been null (unanswered or blank), such as “Married, Insured, Smoker, Conditions – Diabeters, Conditions – Cancer, Conditions – Heart Disease and more.
  • Better Network Error Messages.
  • Fixed an issue with Dependents Counting Double in some queries.
  • Added Premium to Vendor Performance Report and Export.
  • Added AOR to Explore Policies Defaults.
  • Added Next Year to Dropdowns when in December.
  • Fixed Agent Group Not IN Policies Filter.
  • Fixed Issue with Server IP Whitelist before the Multi-server patch.
  • Commission Received Icon was Updated to something less Cliché

New VICIDial Features Enabled (Source) – Some may or may not work with TLDialer Yet – Ask us!

  • User Inbound Filtered Features.
  • VM (Voice Mail) Message Group Features.
  • CID Group Auto-Rotate features, allowing for CIDs to be rotated at set time intervals.
  • Added a new “CallerID Log Report” to the Admin Utilities page.
  • Added CID Group Type of NONE,
  • Added ability to limit the number of voicemail messages played to a single lead per day, as a campaign setting.
  • Added options for survey outbound campaigns to play lead field value audio prompts as well as multiple audio prompts.
  • Added new System Settings option for alternate agent screen recording buttons and configurations.
  • Added Non-Agent API functions “add_dnc_phone” and “add_fpg_phone”, allowing for insertions of phone numbers into DNC lists and Filter Phone Group lists
  • Added a new User option “Additional Status Group” to allow additional call statuses to be used by agents on a per-user basis.
  • Added new HotKey options to send a call to an answering machine message while also automatically dispositioning the call to the agent can move on to their next call right away.
  • Added Campaign option for Dial Timeout Lead overrides, allowing you to set up a Settings Container with values for manual-dial and/or auto-dial lead fields and associated values that will tie to different Dial Timeouts.
  • Added a new Non-Agent API function: “list_info”
  • Added “$webphone_call_seconds” as an agc/options.php option for vicidial.php to allow for webphones to be automatically connected to the agent session X seconds after logging in, instead of requiring the agent to click to call their webphone.
  • Added 4 new options to the Add/Delete DNC Number page for multi-campaign selections in the web admin: – ALL_CAMPAIGNS – ALL_DNC_CAMPAIGNS – ALL_ACTIVE_CAMPAIGNS – ALL_ACTIVE_DNC_CAMPAIGNS
  • Added a System Settings option for second agent screen script tab.
  • Added a Campaign option to not send an agent immediately to the disposition screen after transferring a customer call to a VoiceMail Message.
  • Added campaign feature for VM Message Groups, allowing agents to be able to select a message to transfer a customer answering machine to from a list of pre-recorded messages that can be defined by time-of-day.
  • Added User Inbound Max Calls filtering options. Allowing selective counting of inbound calls excluding calls involving set in-groups, statuses and with a minimum time threshold. Also, new in-group routing options based on last non-filtered call times
  • Added new “EXTENDED RANGE” option to the Agent Screen Time Display campaign setting. It will show per-campaign time stats, non-paused time and totals in the last row.
  • Call Quota Lead Ranking, a completely new and different way to do lead recycling for outbound calling campaigns.
  • Added new Non-Agent API ‘custom_fields_copy’ option to the ‘add_list’ function.
  • Added new Non-Agent API function “update_cid_group_entry”, allowing you to ADD/UPDATE/DELETE or get INFO for the CID Groups or AC-CIDs.
  • Added new options to the campaign Agent Screen Time display setting to allow for agents to see a range of dates for their time stats.
  • USA zipcodes and International phone-codes data files have been updated!
  • Added SIP event logging report, to go with the new SIP event logging experimental features.
  • Added an options.php setting to allow for control of whether MONITOR, BARGE, WHISPER are allowed on the Real-Time Report.
  • Added an options.php setting to add a LOGOUT link on the real-time report. Also the real-time report will now only show the MODIFY and SUMMARY links on if the user has permissions for them.
  • Added “Admin User Redirect URL” settings, allows you to define a web address that a user is sent to when they log in to the admin web screen. Usually used for level 7 report-only users to send them to one specific report when they log in without seeing the admin web screen at all.
  • Added more options to the Campaign “Hide Call Log Info” feature to be able to show only the last X number of calls in the Lead Info screen. Also added a User Override setting for this feature.
  • Added new “Mute Recording Button” feature as a campaign/user option(must be enabled in System Settings first), to allow agents to be able to mute or pause a call recording.
  • Added options to define the static prompts to use for the Inbound Group “Play Place in Line” feature.
  • Added new Agent API function: ‘force_fronter_audio_stop’ will send command to fronter agent session to stop audio_playback playing on it. Will look for other user session currently on a call with the same lead_id that the named agent_user is on the phone with. Works with CCC.
  • Added “Manual Dial Validation” campaign and system settings, forces agent to manually enter in the phone number before a call is placed. Does not affect 3-way calls or transfers.

Patch 7.5.0

Lots of things have changed since the last patch, and we have been slacking on updating our version number! The reason for the jump in the version change is two-fold. TLDialer VICIDIAL Cluster Support, and the rewrite of the TLDSIP Softphone. Most other changes had to do with reports and minor bug fixes as well as quite a few changes and additions to the Lead Post system. Thank you for your patience with us during open enrollment as we have been slammed beyond measure and have been working hard to keep the system running smooth, fast and secure.

TLDialer Cluster Compatibility – Automated Multi-Server / Multi-Phone Management for VICIDial

For our agents running more than 50 concurrent agents on the Dialer, as well as multiple Carriers, Inbound Calls, Dialer Outbounds and tons of transfers, we have added VICIDial Clustering support to the TLDialer system. This required a full analysis and modificaton of changes needed to the TLDialer system as well as a rewrite of the softphone to support registering to multiple phones and managing those connections.

What is a VICIDial Cluster?

A VICIDial Cluster is a VICIDial system that has multiple Asterisk PBX ( Phone System ) Servers connected to one Main Central Database and Application Server, which can also be a Cluster Server itself, and is usually the main cluster. Most all of our TLDialer Accounts are using single Cluster Mode as throwing more “Power” to the AWS Instances we used seem to suffice for 50 Agents or less. Once we hit 50 Agents however, things started to get screwy. We kept getting random Audio Issues, Crashing or One Way Audio on IAX Channel Transfers ( Conferences ), Crashing on the Music Server for On Hold Music, Broken, Seized or Crashing IP Relays (ip_relay) as well as a ton of other Stability issues. The systems didn’t go down, but during peak hours transferring to Verification departments and such was hit or miss, causing issues with call floors.

As you may know, TLDialer uses VICIDial as the Dialing system, but we have an integrated interface that controls and synchronizes data between TLD and VICIDial seamlessly. This is our own invention, and we were hoping to keep to the single server model as long as possible, we however hit our limits early.

Clustering is not simple. It requires quite a bit of configuration, and normally through VICIDial itself it requires you do a ton of maintenance to the Phones and Users to make sure they get logged into the system on the proper Cluster that has the least load. This can be manually done by the Agent who can choose to login to one of X servers, or VICIDial itself can do the load balancing via Phone Aliases, with the Caveat of having to Create and Manage a Phone Per Server per User, and keeping those settings in sync manually. Otherwise you have to change which server the Phone is on at the beginning of the day to load balance based off which of your Staff is actually logging in for that day. It’s a nightmare of management. Enter TLDialer’s Auto Balancing Management system.

TLDSIP & TLDialer Automated Cluster Balancer

Our cluster balancer requires no work on your end. Your agent logs into TLDialer as Usual, click connect and BOOM, they are logged into the lowest load server.

We do this by Automatically Creating a Phone Alias ( EX: 1111ALIAS ) for the Agents Primary Phone (1111), we then Create or Update Multiple Phones that are not the Primary by appending their Server ID’s, which we have Standardized as A1, A2, A3, etc. ( A = Asterisk ). so when you Update 1111 it looks at all the servers available, then creates 1111A2, 1111A3, 1111A4, etc (Primary Server is considered 1111 so there is never an A1 ). It then Creates or Updates the Phone Alias with the list of possible logins for Load Balancing for that Alias. So 1111ALIAS has a Login List of 1111,1111A2,1111A3,1111A4.

We rewrote the TLDSIP Softphone from the ground up, using newer best practices. By doing so we were able to have it manage and handle multiple registrations. Normally on a Cluster Setup, using the Phone Alias Load Balancing method, you would need to go configure all your softphones with the accounts and credentials of EACH Servers Phones that are part of the Alias. You can imagine that this is a bit of a headache, especially when agents play round robin. The More Servers, the more setup. Enter the new TLDSIP.

TLDSIP auto registers to every servers on Login and waits for an Incoming connection. For Blind Dialing it automatically uses connection 1. When an incoming call comes in, it auto accepts and then UNREGISTERS the other Devices. When the Call Hangsup, it Reregisters all the phones to be ready to login to the Dialer Again. As a side effect of this Channel Management, the system can now Re-register and even Reconnect a call if it was lost due to a blip or a timeout, which should increase call stability.

The Ethernet Icons on the bottom left of your screen will show you if your phones are registered or not. They will all be Teal when they are ready, and then Change to Green when in a Call. Unregistered Phones will show White. Phones that should be Registered will try to Reregister automatically if they lose connection.

So as long as you ONLY use TLD to Manage your Dialer Users and Phones, and not the VICIDial Admin interface, these phones will always stay in sync and up to date. You can open the User on TLD, set the Caller ID for Example, and those changes will reflect on all the associated phones in the Cluster. Tada!

Automatic IP Authentication Across Clusters

Yes! This works now too! If you have an IP Unlocked user, or if you add an IP to your authorization table, all clusters should now authorize that IP, not just the main TLDialer server. This makes management even easier!

Update and Changes


  • Now Supports Multiple Registrations
  • New Icon at bottom left which shows you which lines you have avialable and their status. White is Unregistered, Blue is Registered. Green is In Call, Yellow is Unregistered while in a Call.

  • TLDSIP will Auto Register on Load, and Auto Unregister other Phones that are not part of the Current Call when on a Call.
  • Hovering over the Icons will tell you the Server ID and Status.
  • As a side effect of the new TLDSIP, when a blip happens or a timeout occurs on registration, the phone will now auto re-register. This should solve some long standing issues with people on shoddy connections timing out but still on the phone and losing the call.
  • Fixed TLDSIP Not working on iPadOS. Thanks Apple.


  • Monitoring actions now log properly in TLDialer. We have enabled “Monitor Logs” so you can run queries to your hearts content on the Monitor logs.
  • Changed Monitor Icon to look like a CCTV instead of an Eye.
  • Changed Whisper button to look more like an ear.
  • Added Monitor Logs to TLDialer Logs on Contact Form.
  • Fixed bug where address was not being synced to Vici Lead if it existed on creation. We had address instead of address1 selected
  • Fixed Ingroup Ranks not Editing due to limiting to over 100 ingroups. Some of you have way too many ingroups.
  • Added a 5 Second Timer between clicking Monitor in the Live Agents Page to prevent people from double monitoring.
  • Added a 5 Second Timer before Manual Dials to prevent agents from smashing the button and calling the same person multiple times.
  • Fixed an issue with external transfers where it would incorrectly “Remember” the external number set. This did not affect presets.
  • Fixed Set Monitor to search for the agents current conference instead of using possibly stale data from the interface not refreshing. Apparenly Dialer conferences change for the agent when you leave a conference.
  • Added “Show Load” Option for the Real Time Report to monitor all your servers at once when using Multiple Clusters, but this also works with a single server.
  • Fixed an Issue with timezone settings for West Coast clients in the TLDialer System causing initial login to be a few hours behind.
  • On Multi-Cluster setups, TLDialer will autocreate an Phone for every server as well as a Phone Alias for the User. Updates to Phones on a TLDialer User will be copied to every phone they have, but you will be editing the primary phone / extension. We have removed the ability to choose the server the phone is on as we use the Primary server for the Primary phone and the rest is handled by the system automatically.
  • Added Support for IP Whitelisting for multiple servers.


  • Added new “Survey Type”: User Performance Survey
    • This forces the user doing the survey to choose a User they are reviewing which is attached to all subsequent questions.
    • We have added User Reviewed into all logs and added groupings for Users Reviewed in the Survey Report.
  • Surveys can now be limited by role, abilities, user group and user level
  • Fixed Survey Answers for Answer Scale Type.
  • Added First Name and Last name of the Lead to some survey logs.
  • Converted Survey Reports to Data table.

Welcome Emails

  • Emails and Marketplace Emails can now have their individual “From” and “Email” in the Reply To set. This will default to account settings if not set.

Relays, Vendors, Upload & Lead Post System

  • You can now add a Relay Filter so that Relays do not get sent if certain data in a specific field is set with specific values. This only prevents the relay from firing if it comes in via FTP, Post or Upload, this does NOT prevent the use of Relays in Campaigns or via Direct Send. . This is helpful when using a single relay with referrers or campaigns where some should be dialed and some should not.
  • Added “Prevent Converted Phones” from being relayed so you don’t bother your clients when vendors give you bogus data.
  • External Lead Updates can now run Update Lead Status automatically when present to stamp things. This also updates Dialer leads associated.
  • Fix preventing leads coming in with 0000-00-00 or 00/00/0000 for date fields
  • Allow Post System to Skip “Front” status which was causing errors on external update.
  • Reorganized Vendors Data pane to have its own Tabs as it was getting unwieldy.
  • Added Import ID to Post System Error Exports
  • Fixed Validations for Decimals in the Vendor Post system when people were sending things with commas.
  • If a State field comes in and are greater than two characters in length we now grab that data from our meta table and insert the right state. It still needs to be spelled correctly.
  • Added Inbound Price and Inbound Status to override the Default Status and Default Price. This allows you to mix inbound and outbound vendors into a single Vendor Source.
  • Updated Vendor Performance Report with new Metrics, including Fronts to Vendor Performance Report. Should be a lot cleaner now.
  • Vendor Logs got some Love. They now have the option to show what products ( comma separated ) and carriers ( comma separated ) and total sales made on a lead in the Full Report log so your vendors can keep an eye on you if you want. This is useful for Vendors that you are doing revenue share with.
  • Added Passthrough Logs Option to pass the logs for a Vendor into the Passed through Vendor instead of the Passthrough Vendor.
  • Added Custom Status code to Statuses and made these statuses able to send with Relays for external systems that transfer calls that we update status on Disposition.

User Interface Changes

  • Add Click to Copy to the following things:
    • Policy ID on the Policy on the Contact.
    • Name on the Contact.
    • Lead ID on the Contact.
    • Dialer ID’s on the Contact.
  • Now Trimming Name on Leads with extra spaces.
  • Now Ordering Products by Description instead of Name so they should be alphabetized if the default order is set to Description not name.
  • Made Countries order by Description instead of Name,


  • You can now Reschedule CSR’s. The amount of times a CSR is Rescheduled is kept as a counter, as well as by who and when.
  • Renewals vs Rewrite
    • This used to be a Yes or No value but now is No, Rewrite or Renewal.
    • The Contact form has been Updated to state what type it is if selected.
    • You can also search by these types now.
    • Added “Is Renewal” to Campaigns
  • Touches should now work properly. They were always set to 1. Touches means how many times an agent has “Viewed” the page. We went back and resynced all touch values to the total amount of Views in the log for each lead. This took a while but it is worth it. Now whenever you open a lead it will automatically increment the touched by 1. Leads now properly come in as 0 too, they were coming in as 1.
  • Added View Others Notes Ability
  • New Feature: Autocomplete Only Mine or Global Callbacks, to prevent completing callbacks set by someone else automatically.
  • Household Income no longer filters out excluded dependents.
  • Fix to reset commissions paid when recalculating commissions.
  • Added Campaign Checker Agents Available Path in the Egress API. This allows you to use External Systems to Query whether they should send a call or not.
  • Created newer more effective way to serve you JS Without getting you stuck with the cache. Finally! All of our JS files are rewritten based on the version and revision number when we update the codebase, so you are guaranteed to get the latest changes on Refreshing the page.
  • Fixed Explore issue with 7th non-existent column.
  • Statuses on Fields table now shows TLDialer Status and Custom Status Mapped.
  • Made Massive Improvements to Quick Search by adding some new key indexes and querying data slightly different. On large accounts with 2 million+ leads the searches went from 8 seconds to less than a second.

Patch 7.0.6

TLDialer Changes and Updates

TLDialer 2.0

We did a bit of an OVERHAUL on our TLDialer Apps system, which allow us to connect directly into the internals of the dialer and modify behavior and allows us to do things like State Routing. We now have a fully dynamic system, that follows our own patterns and enables us to add new apps dynamically! We were even able to inject code into the main dialer form to allow us to do previously impossible things that have been causing some issues. Along with this have come some significant improvements to the transfer system, barging, monitoring, and more. There is much more to come with this new System! And now for the Updates.

As always when TLDialer Changes, make sure you have your agents all log out and log back in.

  • TLDialer Apps use Routing Extensions to route to each app. The Prefix for apps is TLD{appname}
    • Prefix: TLD
    • App Name: Proceeding 6 Characters
    • Extra / Custom Extensions – Proceeding Numbers after App Name
  • All apps can be duplicated/ isolated and configured by changing the extension to TLD{APPNAM}{EXTRACHARACTERS}
  • TLDialer App Configs can be overridden by creating a fake DID (Ask us to do this) to set DID Custom Parameters on.
    • Each App has different Parameters that go in each custom DID field. Each app has it’s own defaults so most of the time this fake DID is not needed.
    • If you configure one of the custom fields with a parameter, it will override the default. Provide nothing in the custom field and it will fallback to the default for that parameter. Not setting custom fields will just have all parameters fallback to defaults which renders the fake DID fruitless.
    • You can override the default behavior of an app by making the DID Pattern the App name, or you can create your own extension by suffixing anything after the App Name. Just remember to point your Extension to the App with the modified extension you created and it will isolate to just that one.
  • Apps Available
    • TLDSURVEY – Survey System Extension
    • TLDSRVDCL – State Routing Inbound Extension
    • TLDSRVDAD – State Routing Outbound Extension
    • TLDSRXFER – State Routing Transfer Extension
    • TLDBATMAN – No Agent No Answer Transfer Extension ( NANA )

TLDialer General Changes

  • Loading Leads on Calls should now load faster, we are skipping the AJAX Check for the Vici record and taking it straight from the TLDialer Process. Should be near instant now.
  • We now Prevent you from logging into the Dialer if you have another Instance open somewhere else. We now also change the Status to “Duplicate Instance”.
  • You can no longer click the “Disconnect” button, or any buttons but the login buttons while disconnected and have the system attempt to dispo a lead for your disconnected user or some other wizardry.
  • The Starting Status Message is now “Authenticating”. If you somehow click before Authentication or Instance Denial runs, it will tell you to chill out for a second until it says “Ready to Connect”.
  • Once the server Authenticates you do not have another instance running it will now turn blue and say “Ready to Connect”
  • If the server stops sending data to you temporarily we will now block buttons from being pressed.
  • The Emergency Logout button now also Resets the TLDialer Instance of the User being kicked to make sure errything works allright. When being booted, or disconnecting yourself, we now remove the Vici session completely so we can avoid the annoying popup that asks if you want to leave the page or not.
  • Emergency Logout now runs Slay Session, which sends a message up the socket for the agent to navigate away, clearing the session completely.
  • We Removed AGENTDIRECT From the Transfer Menu, it shouldn’t really be there.
  • Added “Client Registered” filter and schema item so you can see if a lead’s phone number is in the client list or not. Added this to TLDialer Leads.
  • Added GMT Offset Filter to TLDialer Leads.
  • Fixed issue with TLDialer Leads Menu not saving correctly on refresh.
  • Added Fronts and Converted Fronts to Agent CPA Report.
  • Added Outbound Autodial field to live agents schema.
  • Updated AMD Rate to be more appropriate to what it actually is. Hover to see the formula! Remember AMD Rates include any status an agent presses that has an Answering Machine flag.

TLDSIP WebRTC Client (Webphone)

  • When using TLDSIP, you now have a Microphone Check Meter show up before you connect, to make sure you are getting some audio! It will go away on login and come back on disconnect.
  • When using TLDSIP We now Prevent you from logging into the Dialer if your Audio isn’t working. The Disconnected Status will be replaced with Audio Error.
  • We now only connect the TLDSIP Phone AFTER Authentication, to prevent overwriting a previous register.

TLDialer Transfers

  • Update: 9/23/2019
    • We are no longer using the button presses to prevent transfers. We are now checking all the lines to see if the agent is in a transfer properly and letting them know if they are. Along with this we added some new features to the Transfer Menu.
    • If the Agent is n Queue or in a Transfer it will show what we mentioned above, however, with the new method it is possible to do 4 or 5 way calls if needed, but we currently have this disabled.
    • NEW External Transfers once NOT Detectable, NOW detectable. Yay! This means that you can do a 3 way call with a carrier or a loved one and it should show that you are in a conference instead of showing nothing.
    • NEW The Transfer menu will now let you know if a Manager has Joined the call (Barge)
    • NEW The Transfer menu will now let you know if a Manager is talking only to you (Whisper)
    • Don’t worry, Agent’s still can’t tell if they are being monitored.
  • Things that this patch should fix.
    • Double+ Transfers (Wrong Conference!)
    • No Client on the Line. (Wrong Conference!)
    • Agent gets Dead / Blank Call. (Wrong Conference!)
    • Manager Monitors, but can be heard. (Wrong Conference!)
    • Multiple Agents Talking to Each Other (Multiple Wrong Conferences!)
    • Can No Longer Hear Clients on Line Suddenly (Wrong Conference! Didn’t Disconnect from bad conference!)

TLDialer State Routing

  • We now have a new Script for creating an Ingroup for Fronters to Transfer to that works with State Routing. It still uses the same State Route Inbound Group, but works properly with Fronters Transferring to the group, as well as flagging the call as to what type of Call the originating transfer came from so we can adjust the Agent CPA report to compensate if we need to.

TLDialer Client

We now have the ability to Inject our own code into VICIDial creating two way communication between the systems in real time. It is SUPER fast. We can now press buttons and close unneeded windows in the client.

Things that this Fixes:

  • Agent Logs In, Presses Pause to Start, and goes into Closer instead of Ready.  This was because there is a screen that needs a submit button in the client hit that automatically times out, but not fast enough for our quicker, less patient agents.
  • Changed “Dispo Pause” method to actually select the checkbox in the Dialer instead of pressing Pause for the agent when they press pause during Dispo. This should now work 100% and no longer possibly grab another call if in queue. Woohoo!
  • More fixes as we catch them in action now that we have been able to identify what has been happening and also have a systemic solution to add fixes as needed.


  • Uploader now shows CSV Encoding Type: UTF-8 or ASCII.
  • Uploads should now correctly count the line items in a CSV Upload.
  • Added Vendor ID to Post System when duplicate checking so you can see if the Dupe was in Same Vendor or not.
  • You can now attach policies, notes, dependents ONLY if the Dupe is in the same Vendor, if desired.

Bugfixes & Minor Changes

  • Added Vendor Public Name to Old Leads Schema so you can use it in the Column Builder.
  • Updated New and Old Policies Schema to support Premuium Actual, Actual Monthly, Actual Initial and Actual Yearly where it 0’s out if the Subsidized Premium is more than the Premium.
  • Fixed an issue with Date Paid and Paid checkbox not setting properly when recalculating commissions received.
  • Commissions Received now using the more improved export window so you can select the columns that get exported.
  • Fixed an issue with Dependent Annual Income Validation when set to 0 not updating so it couldn’t be validated.
  • Fixed an Issue with Fronter ID not being set when Posted in via Post System.
  • We can now push a Popup window with Links or Messages to assist an agent. Useful for live screen debugging since most people don’t know how to type when instructed on the phone.
  • TQL can now Force Index joins.
  • Added accounting role to commissions section.
  • You can now see your TLDialer Seat limit and cannot add new TLDialer Seats when adding users if there aren’t enough.
  • Fixed issue with phones checking TLDialer Seat limit incorrectly.
  • Added change_lead_fronter_id permission.
  • “Owner ID” now checks the assigned_id and if the date_assigned is NOT NULL and is greater than the current date, it will show, this allows you to assign leads to an agent for a later date. Neat!
  • Changed some Layout stuff in the Products table for easier viewing of things.
  • Made it Easier to Calculate CDR Minutes from our Carrier. We will soon release a section where you can see the detailed call log data.
  • Added Plan type to Policy Sidebar. Updated the Icons so they are more even too.
  • Product Export now respects current Query.
  • Added some Missing Filters in Old Policies section: Dialer ID’s, Import Source, Row and ID. Lead ID’s.
  • TQL can no longer attempt to aggregate on aggregated fields.
  • Explore now respects “Export” permission.

Patch 7.0.5

TLD & TLDialer Automated & Self-Serve Whitelist

This is a big one, we have been looking to creating a solution to this problem for a while, and it is finally done! Enjoy the fruits of my sleepless nights!

The “Access” panel has been re-written. We changed the External IP Whitelist Users into a Multi-checkbox so it is easier to select agents as well. It has been Renamed to “IP Unrestricted Users“. It’s counterpart: External Admin Level Limit has been renamed to “IP Unrestricted User Level“. And now onto the Nitty Gritty.

  • IP Whitelist: The IP Whitelist is now a full on table. TLDialer itself, also has a Whitelist. These two Whitelists work in conjunction with one another, and can also be used Separately. Most of what we have done is pretty much automated.
    • The Whitelists are split into Allowed and Revoked, all living on the same panel.
      • The tables can be paginated through as well as searched. Each tab has it’s own search and pagination but all 4 tabs run queries at once when you hit go!
        • You can search by IP Address. Allower Name, Revoker Name, and Description. It will try to match best possible using %LIKE%
    • When adding an IP, you MUST Put something in the Description. You CANNOT Update the description after the IP has been added. We are doing this for tracking and auditing purposes and to keep people honest.
    • Agent IP Restrictions Applies to Agents under the IP Restricted User Level and will allow them to log in.
    • Admin IP Restrictions Apply to Everyone on an Account, EXCEPT for the IP Unrestricted Users. Having at minimum 1 Admin IP Restriction will IP Restrict your whole account for Admins. Admins will be subject to the Agent IP Whitelists but also have the additional benefit of qualifying for the Admin IP as well.
    • IP Unrestricted Users bypass all IP Checks regardless of any IP Restrictions.
    • Adding an IP to the Allowed IP’s Table will also automatically add the IP to the TLDialer Whitelist table by default. IF you do not want that IP Authenticated on TLDialer, you can switch to that tab and Revoke it by pressing the red X.
    • We have Automatically imported all current IP Addresses that were listed in the old IP Whitelist textbox.
    • When you revoke an IP it will end up in the Revoke List which can be searched and keeps a history of what has happened with IP Authentication. This is why we do not allow modifying the descriptions as it would interrupt the history flow for auditing.
  • TLDialer Whitelist: The TLDialer whitelist was a little trickier as each individual Dialer Server is locked out from the world except for the IP’s authenticated.
    • Whitelist Interval – 1 Minute: Adding and Revoking in this list takes about a Minute. When adding, the Revoke button will be a black button with an hour glass until the server authenticates it, it will also do the same when revoking until it finally clears it out as revoked.
    • There are Two Types of IP Whitelists for TLDialer
      • Permanent: Will last until manually revoked
      • Temporary: Will last for a Full day and are cleared at 2 AM Account time. This lets you give someone temporary access.
    • Automatic Whitelisting: If a User is Unrestricted in TLDCRM, we will AUTOMATICALLY add the Users IP Address to the TLDialer Whitelist with a 1 day pass.
      • 1 Day IP Whitelists get cleared every day at 2 AM account time.
      • The server will Re-add the IP if logged into the unrecognized IP the next day on login or access to the CRM if still available.
      • The moment an IP gets Auto-Whitelisted a Popup will notify the User that their IP has been added and should wait at least a Minute before using TLDialer to make sure it has been completed.
    • When you revoke an IP it will end up in the Revoke List which can be searched and keeps a history of what has happened with IP Authentication. This is why we do not allow modifying the descriptions as it would interrupt the history flow for auditing.
    • You can add an IP into the Whitelist that is NOT in the TLDCRM Whitelist manually if you have people connecting to use JUST the Dialer without TLDialer, for fronter operations or specific kinds of management. This should be rarely if ever used.


  • Added a Workaround for not being able to Pause from Dispo Mode. We queue up a Pause if you tried to pause before hand and initiate it right after. There may be an off chance you might get a call within the less than a second that it tries to do this.
  • We now have a reset button that will boot you out of TLDialer, even if you left yourself logged in at home. You know who you are…This also runs on Logout.
  • Pushed a fix to allow Bluetooth Audio Devices and iOS in general to work with the Webphone again. They changed things up!
  • Dead Calls should now show up properly in Live Agents Reports.
  • Added “Gradient Mode” which can be turned on per request for Live Agents Report. Most accounts have this on by default now.
  • Minute Resets will no longer run on Inactive Lists or Inactive Vendors
  • Fixed an issue where a person without a valid phone could try to monitor or barge causing all sorts of craziness.
  • We now check the Client Filter Phone Group when manually adding leads to the hopper to avoid calling clients accidentally when mass inserting.
    • There is a checkbox to override this behavior if you want to add current clients.
  • Users can now see their Outbound CID in the Profile Dropdown. Clicking will copy to Clipboard.
  • Users can now see any Inbound DID’s that point to their User in the Profile Dropdown. Clicking will copy to Clipboard.

Contact Form

  • Not Taken Policy Status can now be hidden.
  • Lead ID now copies to clipboard if clicked.
  • Dialer ID’s now copy to clipboard. There is a button for these.
  • Clicking the Clients name in the Dialer Info Box will now copy to clipboard.
  • Alternate Notes Layou in Options -> Settings -> Contact Form
    • Per Request: It is now possible to show a Top to Bottom layout for Notes instead of the standard Staggered pattern notes.
  • Fixed some issues with fields that should have been disabled but weren’t being disabled after Verificaiton.
  • Fixed an issue with the Dropdown Converter that was “undisabling” Policy records that should have stayed disabled after verification.
  • Fields and Sections can now be force hidden by Products.
    • Force HIDE Overrides Force Show. Always.

Quick Notes

  • Quick Notes has been Revamped, someone found it so we added some more features.
  • Quick Notes can be Added for the Entire Account, if so everyone can see them.
    • A user Can Enable / Disable the Quick Note in their profile or you can enable it for specific users.
  • Show Last Note on Quick Note now works properly. A Little tab on the left of the quick note will let you open and close the last note if there is one. There is also a copy last note to clipboard button.
  • An Icon will appear in the Footer Bar where you can hide and show the quick notes quickly.
  • The Position of the Quick Note, Having the Last Note Open / Close or the Quick Note Itself Open / Close will follow throughout your session so you can keep it where you want.
  • Added a Button to Snap Quick note back to Bottom Right corner.
  • Removed the “Close Quicknote” Button and replaced it with a Minimize Button.
  • Changed Colors of the Quick notes to match your brand colors.


  • You can now copy pictures and such from the clipboard into a Document directly on the contact for for lead and policy documents by highlighting the grey box in front of browser and pressing ctrl+v.
  • Documents now populate a mime_type field for each document so we know exactly what it is.  This may or may not be useful for you but it is useful for us.
  • Documents now have a “Type” that can be set while uploading. This should make identifying a document easier. You can create your own custom types.
    • Document Types can now be required on the lead or on the Policy based off system and custom types.
    • You can now edit a documents type and description from the contact or from the documents section.
    • Public Documents. You can now set documents to be publicly accessible (without login) and for these we will NOT be storing a log. This is going to be useful for hosting content that you need to link elsewhere where there may be high traffic, or for emailers and such.

Lead Statuses

  • Added an Option for both the System as a Whole, or per Status, to require at least one policy on the lead before converting the lead. It will stop disposition from happening accidentally.
  • Lead Status Relays can now have a Vendor Group limitation added as opposed to just Vendors.


  • TLDialer Ingroup Ranks and Grades
    • Ingroup Ranks and Grades can now be editing on the User edit screen in the TLDialer Tab.
    • Ingroup Ranks and Grades can now be Mass Edited in the TLDialer Tab under Ingroups subtab.
    • We added lots of features to filter the ingroups so you can set the ranks and grades quickly.
      • Mass Set Rank and Grade to a specific number
      • Copy Ranks and Copy Grades from another User
      • Set and Copy Functions will only change the ingroups that are currently filtered so you can find the ingroups you want then just set those. So fast!


  • We now automatically append some data if it’s not provided in case you do a “Whoopsie” and need us to modify a certain subset of uploaded leads.
    • We now automatically add import_id to the lead if it does not exist. It will set it to the current Upload ID for easy searching.
    • We now automatically add import_source to the lead if it does not exist. We use the actual file name that was uploaded (not the processed file name) for easy searching.
  • You can now use Excel UTF-8 CSV’s. We remove the BOM Byte for you when we detect UTF-8. Yay! That was really annoying wasn’t it?
  • Added a Link to Leads via Upload ID if the file is a Process file, otherwise it will not be clickable.
  • Removed “Upload ID”, “Reference ID” and “Download” tabs to save space on the table.
    • Replaced with “Action”, “ID” and XID”. Hover over the Columns to see their true name.
  • Added some Color to Upload History Table so you can more easily identify error files or files currently in process.
  • Field Mapping Changes
    • You can now copy the values you are mapping to internal relational fields to the clipboard. There is a button next to each unique value.
    • There is also a copy all button that will give you a list of these values to paste elsewhere. If you hold Ctrl + Click this copy all button it will only copy the unmapped values.
    • Made it so that when mapping products it shows ALL products in your account including the hidden ones, since many uploads of policies are historical products from other system

Campaigns (Relays & Mailer)

  • “Age” campaign types can now be sent out by hour not just day, this gives you a little more fine tuned control to allow you to send an email 2 or 3 hours after a lead is received for instance.


  • Date / Time Filters now support relative Days. When you open the Relative Time Menu, there is an input and a button that says “Days Ago”. This will allow you to check “X” Days Ago. Type in the number of days and press the Days Ago button and it should work.
  • Date / Time Filters now allow you to enter the following format 2019-04-20 and it will auto format to your preferred format set in system settings.


  • Fields now has a Settings Column that has icons.
    • Custom Fields and Custom Statuses have a ton of icons showing you information on the Custom Fields at a glance.
    • Custom Statuses will show a tiny sample of how the button will look, including a quick glance as to what the button will do.
    • Custom Fields will show you the type of button it is including icon chosen or default icon.
    • Other Fields show the settings as a JSON dump. You can click to copy the data if needed!


  • Explore section column builders now include the ability to rename fields when displaying tables. This works now only in the Explore sections and Agent CPA Report for now.
  • Fixed an issue with saving multi-checkbox selections on refresh.
  • Added Right Click Context Menu to Delete Filter Tab.
  • Filter Tabs can be Added. They do not save yet.


  • When creating a new lead either via Manual, by Overriding the Dupe Checker, or via Inbound or TLDialer Inbound, the lead will now be stamped with the following possible import sources
    • manual
      • Manually Input Leads
    • inbound
      • Inbound Leads via Legacy URL Pop.
    • manual_dupe
      • Manually Input Leads that already exist in the system and you unchecked the Create Dupe option if it is available.
      • We made the Button “Create Dupe?” Turn Red now to make sure you read the dang button.
    • tldialer_inbound
      • Inbounds created through the TLDialer System.


  • Moved Ancillary and Fields settings into new “Settings” tab. This section cannot be mass updated yet.
  • You can now Force Hide Fields and Sections per Product.
  • You can now add Custom Field Validation Rules in Products for Leads, Policies and Dependents.
  • Custom Field Validations Respect the Custom Field’s settings.
    • If you Hide a Field from the System or Agents (Forms ) then the field will not be validated.
    • If you set filters for the Custom Fields, they will not be validated if they do not apply to that lead or policy.
    • All Custom Fields will show regardless of status in the Products Validation settings so you can mix and match filters and requirements.


  • Optimized the query that grabs TLDialer List data. Split into 3 different queries to use indexes.
  • Vendors with Public Facing Search capability, now require an Export setting in the Vendor to be set if you want the vendor to be able to export, it is no longer allowed by default on the advanced vendor report.
  • Fixed an issue with Vendor Log Exports. Should work fine now.
  • Changed the “Hover” effect on Response to Click instead, and included the copy paste functions added in other parts of the system. Fixed an issue with Public Logs not Displaying JSON if not logged in.


  • Added “Total Fronts” to salesboard as well as Fronter Commissions
  • Fronter Commissions will total into the Commissions Paid box if visible.
  • You can now also Toggle a Fronter Icon on the User if the user has that role so you can identify Fronters more easily.


Callbacks got some more love in this patch!

  • Callback section had buttons standardized.
  • My Callbacks section had buttons standardized.
  • Option to set Callback as a Global Callback. Users by default can see their own and global callbacks. You can find this in Settings -> Options -> Contact Form
    • When setting a callback, if the system option is set a checkbox for “Set as Global” will appear as an option.
  • Dialer Callbacks Windows (Modal) now has an Icon next to the leads name that shows whether the callback is yours or global.
  • Icon Added to Callbacks and My Callbacks to show whether the callback is assigned or global.
  • Fixed and issue with Untrash and Uncomplete Callbacks when checking for user max settings. Should work properly now.
  • Removed check for max callbacks when callback is global.
  • Autocomplete Callbacks on next dispo is now an option that can be turned off, but is by default on.
  • Fixed an issue with a timer preventing callbacks from being cleared if they were done within 15 minutes. Now it will only not clear the current callback that has been set if it is for today until you navigate away and come back and re-dispo the lead.

Bugfixes and Improvements

  • All user information now copies to clipboard under your username in the top right.
  • Added “Change Agent of Record” permission.
  • Logs with JSON Data Tags (Such as Request / Response) are now Click to open instead of Hover. If you click the data itself in the Flap when it is open, it should copy the data to clipboard for easier access, no more click and drag for copy pasta!
  • Explore section tables have been revamped slightly.
  • Fixed a couple of minor query issues.
  • Fixed default Daily Reset Limit for Lists in Vendors when adding them.
  • Added Total Quotes to Agent Production Report.

Patch 7.0.4

Explore Beta

  • We have added Links to Explore section on the top menu next to Customers. This will allow you to play with the Explore Section.
  • Changes WILL Happen in this section we have a lot planned for it, the goal is for these sections to replace “Leads” and “Policies”.

TLDialer – Multi-Tab / Window Fixes

PLEASE MAKE SURE TO REFRESH YOUR BROWSER TOMORROW! You might not be able to dial until you log out and in as we changed some core parts of how we connect to the Dialer sessions.

One of the issues we see most often is Agents who have access to Live Agents report get stuck because the softphone tries to re-register while they are on the dialer. Unfortunately we have no logs to reject registration or even know the phone is registered.  This patch should resolve a ton of those issues using our own custom internal system. This issue was one of the reasons we suspect sometimes caused crossed calls.

  • TLDialer now uses an Instance ID passed to the socket server to identify the primary tab. The Instance ID is removed from the URL so you can’t refresh or duplicate tabs onto it and cause problems with registrations. You should only ever have ONE TLDialer tab at a time, however TLDialer commands from other windows should still pass through such as dispositions and such.
  • Duplicating a TLDialer tab should take you to the New Contact Page outside TLDialer.
  • Refreshing the TLDialer tab should take you to the New Contact Page, click TLDialer to start over again. This is actually a cleaner restart!
  • Refreshing the TLDialer tab will disconnect your phone AND log you out via timeout or the next time you click the TLDialer Tab.
  • The Live Agents Page will no longer load the Webphone while you are already in TLDialer.

Everything looks mostly the same but we have solved this really irritating technical hurdle due to browser limitations.

TLDialer – Vendor Sources

  • You can now see the following:
    • Reset List Button.
    • List Active / Inactive.
      • Pressing the Button will Toggle the List On and Off.
    • Total Leads in the List.
    • Total Leads Callable Leads in the List. This does NOT mean Dialable based on your campaign settings. Just that the Called Since Last Reset is set to No.
    • Total Leads in Hopper.
  • TLD Reset Lists now respect the new Vici Feature: Daily Reset Limit and will also show you how many times it has been reset.

Custom Fields

  • Custom Dependent Fields have been added, they work like the rest of them do.

Contact Form

  • Added Calculate Annual Income on the Annual Income field. Press it and it will do the hard maths for the arithmetic impaired. This is useful for putting the Monthly Income and Calculating it into Yearly.
  • Added Custom Dependent Fields into Dependents.
  • Added Options in Settings -> Options to remove Trash for Agents ( under level 30, includes Fronters ).


  • You can now see if a lead has Callback set on the Contact form. There is a little ICON next to the Lead ID / Current Status Button.
  • Agents cannot see other peoples callbacks, but you can give them that ability with “View Lead Callbacks” ability.
  • When you disposition a lead, if you have a callback set for that current day, it will auto complete the callback and any expired callbacks so you don’t need to remember to hit the checkbox.
    • Please do not hoard Callbacks as “Expired” This is no longer possible due to this change, you should be rescheduling and organizing your future calls.

TLDialer Leads

  • Added Area Code Filter.
  • Resets now Respect List Reset Max in Vici.

TLDialer Recordings

  • Added Area Code Filter.
  • Added Length in Seconds Filter.

TLDialer Calls

  • Added Recording ID Filter.

TLDialer Dispositions

  • Added Explanations to Special Statuses: DISPO and WAITTO
  • Changed Y / N Columns to Checkboxes and X’s with color.


  • New Validation Option: SSN Must be 9 Digits.


  • Fixed a Bug on Lead Documents where it was always showing the current date as the date uploaded.
  • Changed Group By in Call Logs to CloseCall ID instead of Unique ID.
  • Added City of Birth to Vendor Post Fields
  • Updated Admin Reports to reflect Stages for Sales instead of Statuses.
  • Fixed an issue with Scripts that loaded on Lead Open not replacing text. Works now!


  • Added Dependent Fields Menu
  • Added Explore Menu
    • Added Leads Menu
    • Added Policies Menu

In the Works!

  • ESignature Generator and Signing system.
    • Digital Signature
    • Use Finger for Signing
    • Auto Generate PDF
    • Track Location, IP Address, etc. for legal purposes.
  • SMS with our Carrier to Text Leads in Real Time.
  • More Commissions Modifications.
  • Multiple IVR Survey Options / Chooser.
  • SalesForce TLDialer Connector.


Patch 7.0.3


  • When an agent loads a TLDialer Lead via a Live Call, whether Manual, Inbound or AutoDial we now remove any leads from the hopper with that Dialer / Lead ID.
    • We also do this again just in case when the agent Dispositions the lead.
  • We can now have a Filter Phone Group set for Clients. When a lead status is pressed that has a Vici Status with Sale Flag associated, it will add the phone numbers for the lead into the Phone Group. This phone group can be used as a filter on campaigns to make sure your outbounds do not dial leads with phone numbers in this phone group.

TLDialer – Agent CPA Report

  • Agent CPA Report has been Launched. We are still modifying and making better / more features.
  • You can find this Report under TLDialer -> Agent CPA.
  • The Agent CPA Report shows you mixed statistics between TLDialer Calls and Sales for the Agent.
    • In terms of Cost or Spend per Agent, we use a new table called tldialer_first_contacts which attaches all the information regarding the TLD Lead and Dialer Call and it’s Origin (Inbound / Outbound) to determine Cost based on the First Attempt at a Lead.
  • Some new Definitions:
    • We have added some new fields into the Calls section (Inbound and Outbound Logs) to determine 2 main types of calls per log. These are also reflected in the tldialer_first_contacts log and will now be standard terminology throughout the system for the call logs.
      •  Inbounds
        • Queue Calls
          • The call came in via a normal inbound DID Route
        • Direct Calls
          • The call was directly to the agent via DID or a Direct Agent Transfer ( Agent Direct )
      • Outbounds
        • Auto Calls
          • The agent received the lead from the Auto Dialer Campaigns.
        • Manual Calls
          • The agent found or created a lead and dialed it.
  • Overrides
    • Overrides allow you to customize the report in many ways. You can override the following data points:
      • Most of the following overrides are presented so you can create a more flat rate Agent CPA calculation that is not reliant on actual lead cost or actual call counts.
      • Inbound and Outbound Breakdown Costs per Lead against CPA.
        • This overrides the Cost value that is pulled from the first contacts table (which comes from TLD) and does a flat calculation at that rate for that type.
        • You can set the Rate to 0, so you can simply remove certain call types from the calculations for CPA (such as Direct Calls / Callbacks and Manual Dials ) You can also set rates higher for more important calls such as inbounds.
        • Leaving Settings blank will use Costs gathered in the tldialer_first_contacts table.
      • Inbound and Outbound Weights against Conversion Rate.
        • You can use any number (including decimals) to count a call type more than once. The default weight for a call type is 1.
        • This effects the Closing % Calculations so you can give more favor to outbounds over inbounds for example.
      • Penalty Threshold.
        • The Penalty Threshold is the Target CPA, which then is used to compare to the Agent’s Actual CPA to see how high or low they are from hitting the target.
  • Note: The Agent CPA Report CORRECTLY will convert a State Routed Outbound to Inbound Conversion as an Outbound Call. The First Agent to touch it is on the Inbound because of the route.
  • Coming Soon: We are working on making this report dynamic so you can show, hide and rename field labels as you like.

TLDialer – New Custom Table – tldialer_first_contacts

  • This table keeps track who contacted a lead first and by what method. It has the added Caveat of also including the TLD Lead ID, the TLD Lead Cost and the TLD Vendor ID for later reference!
  • This table can give us sums of call counts broken down by user to figure out who “took” a lead, or at least contacted it first for calculating into the Agent CPA Report. This is a little more efficient than using the TLD Action Logs which was our original method. We had to make this table to get this data as there where multiple pain points trying to query the data live without this custom table, as the data required does not exist in the Dialer in an Easy to Access Manner. With this table we are able to bring you up-to-date information quicker so it can be pulled in near-real time (about 1 minute behind) compared to waiting over a minute just for a report to load and thrashing the logs.
    • Pain Point 1: Two Logs, Which one was the first contact, how much was the lead worth in TLD?
    • Pain Point 2: State Routing: Inbounds are Converted to Outbounds and in this special case it needs to be noted quickly and efficiently.

Service Objects Validation

  • For SuperAdmins, a second icon shows up next to Validate Address, which will allow you to override the address and accept it as valid.

Contact Form

  • New Option to Mask Phone numbers from people with Fronter Role Only. If Mask Phones for Fronters and Agents is set, it will not override this setting. Any user with Fronter role will have it masked.
  • Now possible to Change PCP Office to PCP Address
  • Married is now Marital Status, instead of a Boolean options (Yes or No) you can now choose single, separated, widowed or divorced. The Married field still exists and checks for married = 1 or 0 or null, if married is not 1 (married) then it considered the person not married.

Lead Statuses

  • Custom Lead Statuses can now be set to Unassign an Agent from a Lead.


  • You can now move policies to another lead. This option will be available for superadmins only. It is a button called “Move” on the top of a policy. A popup will ask you to find the lead you want to move to or you can just type in the Lead ID. This will move the Policy, Any action logs associated with it, as well as any Lead Meta (Custom fields, etc), Documents and Survey Answers. It will also log on the previous lead that a policy was moved out, and on the receiving lead that it was moved in.


  • Leads can now have 4 decimal places for price tracking, for those of you somehow able to buy leads for less than a penny. Good for you!


  • You can now mass send emails to Policies as well as Leads.

Commissions Received

  • Now forcing Chargebacks in the Breakdown to be Negative as well as calculating the total amounts based off the negative.


  • Fixed a bug with Updating Lead Costs as an Admin. It was not accepting the pennies.
  • Fixed an issue with CSS in Vendor Forms.
  • Fixed Minor Issue when searching for fields by field and id.
  • Fixed issue for Empty object in Live Agents Table.
  • Fixed an issue with DateTime, Time and Date Custom Fields when using the alternative Date formats.
  • Finally found the cause of the issue sometimes causing Dialers to Disconnect, it was due to AWS’s internal DNS Failing. Gotta love DNS. We now have failover DNS’s that should maintain reliability.
  • Fixed up some Queries for more faster on the TLDialer Live Agents Page.


  • Updated to latest jQuery

Patch 7.0.2


  • TLDialer Live Agents now shows who the Call is in Queue for when it is agent_only ( Agent Direct )
  • Added New Relationship between vicidial_auto_calls agent_only field and user table.
  • NANA – No Agent No Answer
    • Now Supports Transferring after Expiration alongside Busy and Hangup. This allows the phone to “Ring” when an agent is on pause, but still direct to Voicemail if they don’t log in.
    • Now Support Transferring to AgentDirect Overrides instead of just Ingroup.
  • Updated Method to Detect Who is Logged In Via Sockets. Everyone should be default to Sockets. Originally, the Account ID for the User was set based off the User’s account, now it is based off the Currently logged in account. This enables us to use the Socket Method Across accounts with Multi-user capabilities.
  • Call Queue & Direct Queue
    • Added total_direct to both Socket and Non-socket method, which is a count of the AGENTDIRECT calls in queue for the agent.
    • Added queue details back to Socket method so we have details to use later, as well as improved the simplicity and efficiency of the query.
    • Queue Counter will now turn Red when there is an AGENTDIRECT Call for the Agent in Queue.
    • Direct Queue can now create a Popup if configured in TLDialer Settings, letting the agent know they have an AGENTDIRECT Call, and who it is from. This supports Multiple Agent Direct Calls in queue at once. Ask us to Enable this.
    • Direct Queue can now Ring a Phone Ringing sound when the agent is PAUSED and they have an AGENTDIRECT Call, if configured in TLDialer Settings. Ask us to Enable this.
  • We can now remove the Vicidial “Pop” that occurs when a client gets Transferred into your conference. We will be rolling this out as standard for all accounts as it should increase contact ratio’s and prevent people from hanging up once they hear what sounds like a Dialer Transfer.
  • We have replaced the “You are the only person in this conference” with “Welcome to TLDialer. Unpause to get started.”
  • Due to the removal of the Vicidial Pop, On an Incoming Call, Outbound or Inbound, we will play a tone to notify you that your have a call coming in, the client will not hear this tone. Some of you already had this enabled on top of the Vicidial Pop as it was often inaudible or too short for Agents. This will remain the same for those accounts, but still have the Pop Removed.
    • The incoming call sound can be customized, just let us know what sound file you would like to use.
  • We can now send a copy of Recordings to an FTP of your Choosing.
  • Clarified Which Outbounds are Manual vs Outbound Dials in the Logs.
  • We not have an option for a larger TLDialer bar for those with limited visual capabilities.
  • Fix to default user group for mass edit users.
  • Mass Edit TLDialer -> Leads now works properly. We removed ability to change call count as this can cause many problems.
  • TLDialer Hopper
    • This will show you leads currently in the Hopper
    • You can now manually add leads into the hopper from the TLDialer -> Leads screen, with a custom priority. Normally leads go into the hopper as PRiorit 50 or 99 from the Post system, and as Priority 0 from the Campaigns. If you want them called last set the priority between -1 and -99, if you want them dialed right now set it to highest priority, or whatever priority you like.
      • If the lead already exists in the hopper, it will have it’s priority updated to what you selected.
    • Clicking the Leads in Hopper from the Live Agents page should show take you to the hopper and have any campaign filters from the Live Agents page already set.
    • You can now remove leads from the hopper from the TLDialer -> Hopper Page and the TLDialer -> Leads Page
  • FTP Backups
    • TLDialer can now backup your recordings to your own FTP nightly for the previous days recordings. We can also push all existing recordings to an FTP. Ask us to set it up for you!
      • *Note: Pushing all recordings can take some time depending on your rate limit.
    • Supports Rate Limits / second

Custom Fields

  • The new 7.0 Custom Fields now work properly with the Lead Uploader and should allow you to map fields accordingly.
  • Posting Fields should work as well.
  • *Known bug, if your fields aren’t saving, open the vendor map, hit “Save” and it should work again, also make sure the Vendor is set to allow import custom.

Lead Tags

We are finally taking steps towards automation with Lead Tagging. Tagging is a great way to track leads aside from their current status. Tags Normally are not settable by Agents, we make make a special “Tagger” role and Abilities to View, Add or Remove Tags in the future depending on Feedback. The following automation has been added

  • Lead Statuses can be customized to “Add Tag”, “Remove Tags”, or “Drop All Tags” on Press of the Status
  • Vendors now have 4 Multi-Checkboxes for Lead Tags under the Automation Tab. This allows you to add Tags to the Lead on Creation within the following Scenarios.
    • Add Lead Tag on FTP Create Lead
    • Add Lead Tag on Post Create Lead
    • Add Lead Tag on Upload Create Lead
    • Add Lead Tag on Inbound Create Lead
  • Lead Tags Settings are Not retroactive, so you can, for example, have all leads that come in for a month tagged with one lead, then change the tag for the next month for all new leads to be tagged differently.
  • As before, you can also Mass Add, Remove and Drop tags from the leads table
  • You can also filter leads based on lead tags.

Happy Tagging!


  • Added FTP Credentials Meta Custom
  • You can now export field sources a JSON.


  • Found and corrected yet another deep nested bug.
  • Improved system performance and memory usage.
  • TQL Search Boxes should now work! Some just default to primary key search but we are adding more in as we can.


  • Clarified Some of the Uploader Messaging and Tags
  • Uploader now has Filter Select and more robust info on what vendor you are choosing.


  • Fixed a display issue with Terminate date being calculated when the duration is 0.
  • The beneficiaries section now has an “Add Another Beneficiary” button to add another. Beneficiaries by default get 100% of the benfit. If you add a numerical percentage to an additional, it will subtract from that 100% automatically. You can add as many as needed. They will show up before the contingent beneficiary.
  • Added Save buttons in each policy section
  • Moved Copy buttons near save buttons.


  • Minor fix to CSR section, no longer forces all settings even on refresh.


  • Vendors can now have CORS Domain Settings configured when Posting. This allows you to use jQuery or Javascript to post leads into the CRM without that pesky Cross Origin Resource Sharing message.

Commissions Received

  • You will now get an error message when trying to add “First, Full, Last” commission Types if they already exist for a particular policy.
  • When adding “First” commission type, the “Legacy: Paid” checkbox will automatically check itself and it will stamp the Date paid with the date paid on that commission record.
  • When mass editing, recalculations now occur on the policy being edited to, as well as policies that have had that commission removed from them.

Miscellaneous Updates

  • Updated to Latest NodeJS

Patch 7.0.1

7.0 is coming along full force. The remaining issues with the Schemas and Database methods have been ironed out and we should be ready to launch the explore section shortly once a bit of interface work is done.

If you love TLD Please, please please give us a kind Review on G2 Crowd here! We are 4 shy of enabling another category on their platform. It will help us grow, and in turn, help you grow.

TLDialer Updates

  • New Option: Notify Agent when an Agent Direct Call is in Queue. A popup will open if a Call enters an Agents Queue and they are either on a Call, Dispo or Pause, notifying them that they should leave pause to grab the next call as it is Directly to them. The Queue Icon will also turn Red while the call is in the Queue and go back to blue once it either drops or is transferred to the Agent. Ask us to Enable it if you like!
  • Added new VICIDial Admin button in the TLDialer Sidebar.
  • When you click your Name in the Top Right, if you are a manger or higher you will see your Password among your credentials, Click the password to copy it so you can more easily log into the VICIDial Admin Panel. Unfortunately VICI uses PHP_AUTH which requires you to enter in the username and password manually, we are investigating workarounds, but aside from creating a separate script that lives in Vici, we have had no luck after trying Various methods ( Dang you CORS!!!! ) We try not to touch the VICIDial Codebase to keep it pristine and workable for Open Source Releases and Updates.
  • If you are using TLDialer and you Clone a Vendor, a List will now automatically be created based on the Cloned Vendors list but with the New Cloning Vendors List ID, if it has one attached. This is similar to adding a new vendor, just a little more automated.
  • Added Lead List Active to TLDialer Leads Filters.
  • Minor Fix to Vendor Meta. Trimming Values so that we don’t have phone numbers with spaces on inbound_phone causing numbers not to detect and join for easy management.
  • Fixed an Export Bug in some TLDialer Sections.

TQL Updates

  • Fixed an ordering issue with Join Columns by using Hashes. Faster and More Efficient now.
  • Fixed a Join Ordering Issue with similar methodology, also faster and more efficient.
  • Created TQL Debugger to both Log Queries when needed, as well as instantly notify us via Slack whenever a query goes wrong so we can fix it ASAP. THis has allowed us to correct the last few minor kinks during the soft launch.

Custom Fields

  • Lead Post System now works with new Custom Lead Fields including Mapping and Uploads.
  • Custom fields can now be hidden based off of Lead of Policy Data and Criteria.
    • Criteria Includes: Lead Status, Policy Status, Lead Tags, Policy Tags, Vendors, Vendor Groups, Carrier, Product, Plan,
    • Every Filter Option also has an Option to show on a “New” lead, regardless of Filter Criteria.
  • Custom Fields can now be shown or hidden based off of User Roles, Abilities, Levels and User Group Membership.
  • Custom Fields will no longer write to DB if they don’t have a previous value and no values are provided. This will save us some space. If you write to a custom field and then delete the value, it will keep the meta record, but null out the value instead. Yay for saving space!
  • Removed Remnants of Old Custom Lead Model.
  • Disabled old Custom Meta section in Leads, prepping for Explore Section to Launch this Weekend if possible!

IVR Surveys

  • Refactored / Removed Auto-Fill of Values in IVR Questions and Surveys, and show Placeholders instead, making it easier to customize. Why store the data unless we are going to use it?
  • Added More Default Prompts to IVR Presets and Survey IVR Configs.
  • IVR Surveys now have a True Final Fallback for all Options.
  • All Prompts and Options related to the IVR Have Icons Indicating what the Fallback is, and if it is selected.
  • Added Missing Prompts for Resume, Pause and Invalid Entry.
  • Start, Stop, Restart, Reset, Pause and Resume should now all work properly in all cases, this includes while a question is being read and while an answer is being received / spoken by the client. Resuming will repeat the current question.
  • Added a Cleanup Process to the IVR in case of an unfortunate disconnect. Should prevent some Recordings from continuing longer than they need to.
  • Fixed a bug that was not resetting the Total NoVoice Retries, causing them to accumulate and then fail.
  • Added IVR Voice Silence Timeout, which was causing problems as the system could wait infinitely if it did not detect silence. This has resolve most remaining issues along with the retries fix.
  • IVR Fallbacks for Prompts and Settings follow an Order from First Chosen to Last. For example, if you change the Survey’s settings, it will use that value, but if you change a questions setting it will use that value instead. If you have no value set, it defers to the system preferred default.
    • The Fallback Cascade Pattern:
      1. Question
      2. Survey
      3. IVR Preset
      4. System Default
  • Added Some Product Filters to be able to Search for IVR Enabled Products based off a couple different settings: Presets and Surveys Selected.

General UI  Updates and Bugfixes

  • Fixed some Icons on Custom Field Types so they should all show now.
  • Updated to latest FontAwesome Pro Fontset. More Icons!
  • Added Some more Instructions and examples in the UI Help for Multi-Checkboxes. Learn to use them they are a timesaver!
  • Added Ability: View Admin Lead Data
    • This allows a user to see the bottom left panel admins can see related to a lead, who created it, converted it, the vendor, etc. This section is very very ugly and outdated as it was originally used for Debugging information and I had no idea anyone was actually using it.
  • Fixed an issue with Mass Update Products where it wasn’t clearing settings when trying to clear.

Patch 7.0.0 – Soft Launch

Wednesday we Soft Launched TLD 7.0. This was a Significant Update and Rewrite of some of our core backend processes which will allow for further and faster development in the future. Some of the new features are still hidden since they are still being tested, like the new Explore Section which will replace the Leads and Policies section ( But won’t be removed just yet due to legacy preferences ). Below you will find a massive list of changes, updates and bugfixes that have been going on since our last patch. We held off on publishing these notes as we finally felt we were nearing the finish line for 7.0.

7.0 Major Changes

Explore Section

The explore section will come out in the next patch, as it is a major change to how data is looked through. It works similarly but there are more options and more room for information. There are many more ways to customize the system as well in this new revision! We know you will like it!

Custom Fields

  • Custom Fields have been moved into the Fields Section. The Custom Leads Fields link has been updated.
  • Custom Policy Fields are now possible to create!
  • Custom Fields can now be Filtered by the Following Permissions.
    • User Level
    • User Group
    • User Roles
    • User Abilities
  • Custom Fields can now be Filtered by the Following Filters
    • Leads & Policies
      • Vendor
      • Vendor Group
      • Lead Tags
    • Policies
      • Policy Tags
      • Carriers
      • Plans
      • Products
      • Stage
    • Filter Policies also have a flag for whether or not it should show up on “New” records or not.
  • All Filters and Permissions include “Not In” as an option. EX: Everyone BUT Agent Role.
  • Removed Type: Checkbox
  • Added Types: Boolean, Alphanum Select
  • We have Migrated all your old Custom Fields into the new system automatically, nothing should be need to be done by you.

TQL Changes

  • The TQL Core has been Partially Rewritten
  • We can now import to further depths without running into recursion issues.
  • We now exclude the Key on the imported table that was imported since it is a duplicate and not needed
  • We now have support for SUM, AVG, MIN, MAX, GROUP_CONCAT fields automatically built in for every single field.
  • Caching Schemas has been Improved.
  • Fixed some Join Reference Issues incorrectly placed in Schemas.
  • Improved Memory Consumption and Processing Time for TQL Schemas by 300%.
  • Created a new Internal Debugging Tool for TQL that Integrates with Slack and will notify us of anything Fishy going on in real time. It’s super robust, too bad you will never see it! But trust us it is useful! We are notified of errors in real time without your input, so we can take action and fix quickly.
  • TQL DateTime method has been improved. Now using much faster query formats for searching between dates. Previous version was not using Indexes in certain instances causing severe lag locking tables.
  • TQL TLDialer – Vicidial Schema Updates
    • Added Relationships to park log for users, and custom time queries.
    • Added relationships to agent log for campaigns, also added custom time queries for pause time, talk time, dispo time and dead time.
    • Added queue time to closer log.
    • Added Campaign Filter to Live Agents Table.
    • TLDialer Leads now has “Entry List ID Filter” as well as “Actual List ID Filter” to work with Cascades more easily. Actual List ID will do a Search in Entry List if its not 0, or the List if it is, so you can track where your leads originated from.
    • Added Source ID Filter to TLDialer Leads.
    • Fixed a Query issue with the TLDialer Resets, we needed to use :59 seconds instead of :00 for certain parameters. Thanks for the report!
    • Fixed another Query Issue specific to Last Local Call Time. Apparently, Vicidial stores the DateTime Called in the TIMEZONE OF THE LEAD, based off the GMT Offset Now Field. Why? I don’t know. We changed all Local Call Time queries to now convert the current time period and the last call time to GMT before comparison.
    • DID’s section has gotten some updates, new filters, and more relational data to join together.

Other Changes and Fixes since Last Revision

Contact Form

  • Policies can now have Custom Fields Show
  • Lead Income & Policy Application Number label can now be renamed in Settings -> Options -> Contact Form
    • These Rename features are stopgaps. We recognize the desire to name fields as you desire and are working on a solution.
  • Added more Removable Fields from Lead Form.
  • Made it so that the Pricing Module will no longer effect Verified Policies. If a Product had a Price Change, the Module was overwriting the initial value causing confusion.
  • Fixed an issue with Appointments and Callbacks where you couldn’t set one after a manual entry of a lead.


  • Vendor List should now show the Colors selected for Labels if the vendor has a public label set.
  • Removed some Fields from the Vendor Print Instructions, particularly “Lead ID”, because when Lead ID is passed, it ALWAYS triggers a Dupe check by Lead ID, as it is a system field.
  • Fixed an issue where Cloning Vendors would copy Cascades and Resets, causing all sorts of problems, we will revisit this later and make it work.
  • Fixed the Behavior of the List Forms Disabling when you change the List ID associated with a Vendor, if you put it back to what it was it should un-disable it.
  • You can now sort Vendors by some criteria.


  • Fixed an issue with Custom Statuses not showing up in Policies section
  • Added “Agent User Group” as Filter and Columns in Policies section.


  • Changed the Way Callbacks” Expired and Upcoming” filter works. They now only show things that aren’t trashed or expired.
    • Added “Abandoned” to Callbacks Filters as well to show you Callbacks set for agents that are no longer active.

Uploader, Vendor Logs & Post System

  • When Uploading Leads and seeing the “Process Upload” panel, if you change the Vendor Selected, without hitting “Select Vendor”, the Process sidebar will be removed and ask you to press “Select Vendor”. This is to prevent someone uploading leads,  having matching columns, and then trying to select a different vendor and process it.
  • Fixed an issue with Post Update Leads. It will no longer accept blank data values, they get filtered out before updating.
  • Changed the way we log data in the Vendor Logs, now a little more robust and clear as to what is what. Added Details section to POST with the Method and Format of the data received.
  • Added Ability to allow Vendor Logs to Play or Download Recordings for the Leads they can show. Lots of protections to prevent unauthorized access have been added. Only Works when Advanced Vendor Report is enabled.
    • You will see a button / Box with the number of recordings on a lead. Click this to open the public recordings page.
  • Added Status Filter to Advanced Vendor Report.


  • Fixed odd case where Updating Users would not refresh Live Agents Table.
  • Standardized Refresh Live Agents to use the Same Function. Now respects Intersection between Campaign and User Ingroups every time.
  • Added Legend for the Columns to TLDialer Dispositions Page.
  • Fixed an issue when Converting the First Policy and having the Lead Status Update. It will not properly disposition the Dialer Lead when you hit Sale for the first time.
  • Cascading and Minute Resets: Now only 1 Criteria at minimum is required to run, date criteria no longer required, it can run purely off the total call attempts.
  • Fixed an issue related to Cascades. If the Entry List ID is not 0 and different than the list_id, it should use the Entry List ID for creating new leads.
  • Added Lead ID and Log ID Filters to Vendor Logs
  • Fixed an Issue where Marking a Policy as “Sale” would hangup the call if Hangup on Dispo was Set and it was the first conversion on a Lead. Whoops!
  • Contacts now have a “TLDialer” tab in the Log section if capable. You must be manager or higher, or have View TLDialer Logs Permissions to see it!
    • Unlike other log sections, this one only loads once you click, we may move the rest of the logs to this method to reduce initial load time on the page for contacts with lots and lots of action logs, notes, etc.
    • This log section mimics the VICIDial Lead Edit / Log section, with some improvements. It is a bit more verbose. It also includes some translated fields such as length of call into a time format instead of seconds. It also includes The TLDialer Reset Logs so you can see how many times the lead has been reset through our custom implementation.
    • The first tab is known as the “Timeline” tab, which is an aggregate of all the logs in order of occurrence with the most pertinent information for the section shown. This is Unique to TLD! Enjoy!
  • New NANAnswer (No Agent No Answer) capability for TLDialer / Vicidial. What this does is check if you have any agents live and in READY or CLOSER mode, and if there are no agents available it will ring and then hangup or give a busy signal WITHOUT answering the call. This is useful for Live Transfer Vendors that want to send their calls elsewhere if you have no agents available. Normal behavior of Vicidial is to auto answer all calls and place them in queues, voicemails, drops, etc. This is custom functionality we have developed. We also have a queue log for information as to what happened with this table.
  • Added DID ACCID Installed Script, so we can do this much faster now.
  • Changed Label on User TLDialer Groups to Inbound Groups / Closer Campaigns for clarity.
  • Working on a new Hangup script to overcome some issues found with Vicidials native API Script hanging up wrong lines. We have a dropdown with current connected lines but it is only possible to hangup blind conferences from there at the moment.


  • Fixed a Display Issue on the notification when adding Inactive or Inactive Visible Users.
  • You can now mass edit users to be AoR as well as Max Callbacks and Primary Language, as well as Disable Global Quote tools and Disable Edit Notes.
  • Users can now have “Editing” Notes disabled. This prevents the Agent from editing any note. They can only add new ones. Maybe this will help with those who are doing the wrong thing and trying to hide something? Although the Logs keeps track of every note change done, this method will make it harder for an agent to goof up and not take responsibility.
  • Agent of Record can now have a State Filter applied so they only show up in the proper states.

Dashboard & Reports

  • Added a couple missing links to the Admin Dashboard for Primary Ancillary and Sale Totals.
  • Added Sort by Premium, Primary Premium, Ancillary Premium to Sales Board


  • Fixed an issue with non set statuses in Email and Relay Campaigns.


  • Added new features to the View Log.
    • You can now see where the lead was opened from (How)
    • You now have a full URL and IP Log of where the lead was opened from.


  • Massive Fixes to Mailer
    • Apparently, Bounce, Complaint and Delivery Notifications have been broken for a very, very, very long time, as most people don’t often use the mailer.
    • Improved the Bounce, Complaint, Delivery Notification Performance and Date Logging.
    • Fixed the Opened Pixel to work again & Improved the Open function performance
    • Fixed Unsubscribe Link to show Email in the TO if a lead Email is not present.
  • You can now Mass Email from the Lead Screen. This works similar to the Relay button.
  • You can now have an Email sent to your client when a lead is received via Post, Upload or FTP separately.
  • You can now have a Notification Email sent to you or someone else when a Lead Arrives
  • All Emails require you to configure and use a Mailer Template if you want to send fields with the emails. These templates need some revision in the future.

IVR Surveys

  • If your account is flagged to use the IVR and has a Google API Key Installed, then you can now “Preview” every Setting and Survey Question, as well as Logic Alternative Questions, straight from the CRM! This way you don’t have to go through your entire IVR Survey just to hear how it sounds.
    • You can even DOWNLOAD the recording! We can now use this feature to automatically generate call menus and voicemail recordings for you! It’s super cool! There are many voices to choose from and inflections, etc. can be controlled via SSML right in the text.
  • Updated Formatting on Survey Inputs for IVR.
  • Refactored Fallback Settings Config for IVR Surveys.
  • Got Recordings Working, they are prefixed with ivr- in the filename when they show up. Vicidial handles the mixing, etc, they can be downloaded and played just like normal Vicidial recordings.
  • On successful completion of IVR Survey, now will pass recording_id to the record_id of the policy, and auto verify the policy. The Multi-User editing system takes care of refreshing the policy for the agent.
  • Survey Logic Sets now Support their own Set of Routes! This way you can route around a question that may have no answer if the Criteria is True.
  • Survey is Live and working!

Commissions Received

  • Fixed an issue with commissions received when recalculating to 0.
  • Recalculate all no longer has limit.


  • Fixed an issue when Searching by email for Policies in Quick Search
  • Fixed an issue when Searching by Two Names for Policies in Quick Search
  • Added “Name” + “State” to Quick Search. You can use a Capitalized State along side the name of the person you are looking for.
  • Added Help Icon to Quick Search
  • Multi Checkboxes can now search for multiple criteria by using commas.
    • Example: You want to see FL and CO, normally typing FL CO would only show an item with both FL and CO, now you can do FL,CO and get both!
  • Fixed an issue with Paginate All


  • Fixed a Minor Issue where ServiceObjects would report the Address was Valid, but it was either not, or there was an error saving data into the ServiceObjects resultset. This was preventing the Validation system from moving forward.
  • Fixed another weird case with the ServiceObjects API.