Healthcare Software Systems: Requirements and Evaluation Checklist
Healthcare software systems are the applications and services that support clinical records, patient access, scheduling, billing, information exchange and operations. Evaluate them as a connected workflow: which system owns each fact, which people can act, what crosses each boundary and how unfinished work is recovered. A feature in one application is not proof that the complete process works.
Consider an appointment moved in a scheduling application. The portal may still show the old time, the notification service may have failed, and an interface may retry an event that was already accepted. Each product can display a reassuring local status while the person attending the appointment has the wrong information. The buying question is not simply whether the software has scheduling. It is whether the combined system preserves the right identity, current appointment and unresolved communication.
This guide focuses on those system boundaries. For the broader product categories, buy-versus-build decision and general selection scorecard, use the healthcare software guide. Here, the deliverables are a traceable requirements matrix, fixed fictional test pack and an acceptance record you can adapt to your own landscape.
Sources checked October 5, 2026. Published by Vero, a documentation-software provider. Examples are fictional. Clinician and health-IT review: pending.
Map the systems and their authority
Start with one episode of work rather than an inventory of brands. Name the initiating event, authoritative source, receiving system, accountable role and completion evidence. Include the applications outside the main EHR: portals, messaging gateways, identity providers, interface services, document stores and operational queues. A manual reconciliation step is part of the system, even when it appears on no architecture diagram.
In our fictional appointment landscape, the patient registry owns the identity crosswalk, the scheduler owns appointment status, the portal displays the current appointment, and a work queue owns unsuccessful communication. The notification service owns its delivery status—not whether the patient understood or agreed to a change. No single “done” field should stand in for all these states.
Scroll horizontally to compare source authority with completion evidence.
System / role | Authoritative information | What proves completion? |
|---|---|---|
Registry / registration lead | TEST-A maps to P-100; TEST-B maps to P-200. | Verified identifiers, not matching names alone. |
Scheduler / scheduling lead | Old A-100 cancelled; replacement A-101 booked. | Readback of both appointments and their association. |
Portal / application owner | A display derived from the current scheduler state. | Correct replacement time; old slot not shown as active. |
Work queue / coordinator | Notification failed; communication remains outstanding. | Assigned exception and subsequent verified outcome—not a transport receipt. |
The 2025 ONC SAFER Guides address EHR system management, patient identification, organizational responsibilities and contingency planning, among other domains. They provide a foundation for identifying responsibilities and failure paths. Vero’s scheduling map applies those principles to this example.
Give every important state an owner
Record where a correction starts and which systems must receive it. If a phone number changes in registration, establish whether the notification system reads that value immediately, uses a cached copy or requires a separate update. Define who notices when the copies disagree. Do not assume that a shared login means shared data, or that a common database means every module uses the same completion rule.
This map also separates an application requirement from a workflow requirement. A note-drafting workspace need not own appointment booking. The scheduler need not author a clinical note. But the organization must still establish an accepted route between the relevant tools and the system of record. Assigning responsibility elsewhere is valid; leaving it unowned is not.
Write requirements at the boundaries
A useful requirement states an operation, precondition and observable result. Replace “seamless integration” with “after a verified cancellation, the portal displays the replacement appointment and a failed notification remains assigned to the coordinator.” Attach a stable requirement ID and retain the evidence needed to judge it.
The matrix below uses the same fixed episode for every provider configuration. It identifies the system boundary and accountable role, rather than asking each application to perform everything. Core checks apply to the complete scheduling workflow. The billing interface is optional because the example can be evaluated without procuring billing functionality.
System-boundary requirements matrix
Vero requirements worksheet
Record each test’s status and evidence. Start with Not run; exclude the optional billing check when it falls outside the agreed scope.
Scroll horizontally to compare the fields. Keyboard users can focus the table and use arrow keys. Record product versions, configuration, test date and internal evidence references in your approved workspace.
| Requirement / test | Boundary / owner | Expected state | Evidence to retain | Assessment |
|---|---|---|---|---|
| Identity survives every boundaryHS-02 · Core workflow | Registry → scheduler → portalRegistration lead + integration team | TEST-A/P-100 and appointment A-100 remain associated; TEST-B/P-200 is rejected. | Source and destination identifiers, mismatch log and no-write evidence | |
| Appointment and communication states stay separateHS-01 · Core workflow | Scheduler → portal → work queueScheduling lead | A-100 is cancelled, A-101 is booked, and failed notification remains assigned—not completed. | Both appointment records, communication state and named queue owner | |
| Transport acknowledgement is not task completionHS-03 · Core workflow | Connector → receiving applicationInterface owner + scheduling lead | An accepted message does not close the task until the receiving business state is verified. | Message acknowledgement, destination readback and completion rule | |
| A retry does not create another appointmentHS-04 · Core workflow | Sender → connector → schedulerIntegration team | Replaying event E-101 leaves exactly one A-101 and one owned notification exception. | Event ID, retry log and destination counts before/after replay | |
| Older events cannot restore cancelled workHS-05 · Core workflow | Scheduler → downstream displaysIntegration team + application owner | An old A-100 booked event arriving after its cancellation cannot restore the old slot. | Source versions, event order and final authoritative state | |
| An unauthorized account cannot change the bookingHS-06 · Core workflow | Identity provider → API → schedulerSecurity lead + application owner | A read-only test account cannot write; denying the request leaves both appointments unchanged. | Role configuration, denied request and destination no-change check | |
| Downtime recovery reconciles manual workHS-07 · Core workflow | Downtime log → recovered systemsOperations lead + IT support | After the approved manual path, reconnect without losing the cancellation or duplicating the replacement. | Manual log, recovery reconciliation and outstanding communication owner | |
| An export reconstructs the episodeHS-08 · Core workflow | Current systems → independent readerRecords lead + technical reviewer | Read both appointments, identity associations, event times, audit history and the open communication task. | Export manifest, independent readback and exclusions list | |
| Cancellation does not become a completed serviceHS-09 · Optional billing interface | Scheduler → billingBilling lead | If billing is in scope, the cancelled slot does not create a completed-service or charge event. | Billing destination state; otherwise an agreed scope exclusion | |
| A partial reschedule stays open for recoveryHS-10 · Core workflow | Scheduler cancellation → replacement request → portal → recovery queueScheduling coordinator + integration team | A-100 stays cancelled; failed request B-101 creates no booked A-101. The portal says rebooking pending and one owned R-101 recovery task remains open. | Cancellation readback, failed booking response, absence of A-101, portal state, recovery-task owner and authorized retry/retest record |
Pass 0 · Fail 0 · Not run 10 · Excluded 0 · Unresolved 0.
View the copyable evaluation record
Copy the complete template into your approved workspace. It includes configuration, expected results, required evidence, findings, remediation, retesting and approval fields. Load the example assessment to see one filled HS-10 row. Complete findings and evidence references in your offline copy.
Reloading or resetting clears your selections. Copy the record to retain them, then store findings and evidence in your approved workspace.
The worksheet starts at Not run. Its example button loads the filled HS-10 assessment from the partial-success case below; other core checks remain unrun. Record Pass after reviewing the test evidence. Not applicable needs an agreed scope reason. A requirement assigned to another system remains a requirement of the complete workflow.
The copyable record preserves every expected result and evidence requirement. Its header has separate fields for products and versions, interface configuration, environment, account roles, test date, operator, fixture and evidence location. Each row adds actual findings, defect and impact, remediation and owner, retest evidence, and approval scope/date. Select statuses here, then complete the findings and approval fields in your offline copy inside the organization’s approved environment.
A reproducible appointment-rescheduling test pack
Fixed facts and expected states
Run the fixture with synthetic accounts and appointments in an authorized non-production environment. Use the same time zone in the source and receiving displays: America/Toronto, UTC−04:00 on these dates.
- Registry: TEST-A/P-100 and unrelated TEST-B/P-200.
- Old appointment A-100: TEST-A/P-100, October 8, 2026 at 10:00; initially booked.
- Replacement A-101: the same person, October 12 at 10:30; initially absent.
- Approved change E-101: entered October 5 at 09:00, cancels A-100 and books A-101 under the supplied administrative authorization.
- Connector: send the change to the portal display and notification service.
- Controlled failure: notification delivery returns failed. The supplied local rule routes unresolved communication to the scheduling coordinator; it does not mark the person informed.
For HS-01, the expected destination has one cancelled A-100, one booked A-101, the replacement displayed correctly and one assigned communication exception. Patient communication remains outstanding. Other cases change the specified failure injection; HS-10 deliberately prevents the replacement from being booked. The test scope is administrative rescheduling and communication.
A fluent completion message can still be wrong
The invented dashboard proposal says: “Rescheduling complete; patient informed; close all work.” Its appointment fields are correct, but it collapses a successful booking change and unsuccessful notification into a single completion claim.
Show the completed HS-01 example assessment
- Evidence supplied
A-100 cancelled; one A-101 booked; notification failed; no completed communication evidence.
- Requirement findings
HS-01 state separation: Fail. HS-03 completion rule: Fail. Other core checks: Not run. HS-09 billing: excluded from this agreed scheduling scope.
- Repair
Keep the booking change. Replace “patient informed” with “notification failed; coordinator to resolve.” Reopen or create the owned communication exception.
- Acceptance decision
Hold acceptance of automatic closure until the completion rule is repaired and retested. Correcting this one dashboard message does not prove prevention.
Run the same controlled failures
Record the exact products, modules, versions, interface configuration, accounts, devices, time zone and date before execution. Restore the fixture before each case so a preceding test cannot determine the next result. Preserve the raw response and destination state before anyone repairs it.
Each case supplies a controlled change and an observable acceptance result.
Case | Change to the fixture | Required result |
|---|---|---|
HS-01: failed notification | Run E-101 with the supplied notification failure. | Correct appointments; communication outstanding and assigned. |
HS-02: wrong identity | Change the event’s portal mapping to TEST-B/P-200; keep TEST-A in the scheduler payload. | Stop the inconsistent operation; neither person receives the other’s appointment. Retain mismatch and no-write evidence. |
HS-03: accepted, not applied | Return a connector acknowledgement while withholding the portal update. | Portal synchronization remains pending; acknowledgement does not close the business task. |
HS-04: uncertain response | Apply E-101, drop its response, then replay the same event identifier. | Exactly one replacement appointment and one unresolved communication task, with retry evidence. |
HS-05: old event arrives late | After cancellation, deliver the earlier A-100 booked state. | The old slot is not restored; preserve the source version and reconciliation decision. |
HS-06: permission denial | Attempt E-101 through a read-only synthetic account. | Denied write and unchanged appointments, not just a hidden button. |
HS-07: downtime and return | Disconnect the portal; use the approved manual log, then reconnect and reconcile. | No lost or duplicate booking; communication ownership survives recovery. |
HS-08: exit readback | Export the episode and open it with an independent reader. | Reconstruct identity, both appointments, timestamps, history and unfinished work; list absent fields. |
HS-09: optional billing | If contracted, inspect billing after cancellation. | No completed-service event from A-100. Otherwise record the agreed billing exclusion. |
HS-10: partial rescheduling | Commit A-100 cancellation, then reject request B-101 for replacement A-101 before any booking is persisted. Hold confirmation notifications. | A-100 cancelled; A-101 absent; portal rebooking pending; one R-101 recovery task assigned to the scheduling coordinator. No rescheduling-complete claim. |
Providers may implement these outcomes with event keys, lookup-before-retry logic or another supported mechanism. Ask them to explain and demonstrate failure behavior. Where direct outage injection is impractical, agree a test-harness method and record the test conditions and any unrun cases.
HS-10: When cancellation succeeds but rebooking fails
Restore the original fixture: A-100 is booked and A-101 does not exist. For this case, replace the notification-failure injection with a booking failure. Allow E-101's cancellation to commit and verify A-100 is cancelled. Then have the authorized test harness reject replacement request B-101 with a configured failure response before creating A-101. A raw failure response alone is insufficient: read the scheduler back to establish that no replacement appointment was persisted. Keep confirmation notifications on hold while rebooking is unresolved.
The supplied recovery rule preserves the completed cancellation, assigns failed rebooking to the scheduling coordinator and prevents an automatic claim of completion. It does not silently restore A-100 or guarantee the requested replacement time is still available. Those are separate authorized decisions. Retain the cancellation result and failed booking request as distinct records linked to E-101.
Required readback for HS-10.
| Record / destination | Expected result |
|---|---|
| Appointment A-100 | Cancelled, with its event association and history retained; not shown as an active slot. |
| Replacement request B-101 | Failed request for A-101 retained in the request/error record; no booked A-101 exists. A requested time is not a confirmed appointment. |
| Portal | Shows the old appointment cancelled and “Rebooking pending—scheduling team reviewing.” It does not display October 12 at 10:30 as confirmed or say “Rescheduling complete.” |
| Recovery task R-101 | One open task assigned to the scheduling coordinator, linked to E-101, A-100 and B-101, with the failure, next action and agreed escalation/absence coverage. |
Next, exercise the fixture's authorized recovery route. Read the current appointment state and availability before retrying through the supported duplicate-prevention mechanism. When the harness permits creation, require exactly one A-101, its correct association and a portal readback before closing R-101. If notification then fails, retain a separate owned communication task: repairing the booking does not establish that the person was informed. Preserve both the before-recovery evidence and the later outcome rather than replacing the original failure log.
The worksheet’s filled HS-10 example evaluates a flawed proposal: A-100 is cancelled and A-101 is absent, but the portal claims completion and there is no owned recovery task. It records Fail, the discrepancy, example evidence references, assigned repair work, an unrun retest and a Hold decision. Load the example and copy the record to see how one complete row connects findings to remediation and approval.
What healthcare software providers document
Use public documentation to select operations to test, then resolve access, licensing and configuration with the supplier. The examples below cover different system boundaries. Sources were checked October 5, 2026.
Compare the documented operation and unresolved boundary—not brand size.
Offering / boundary | Vendor-documented finding | Site-specific checks |
|---|---|---|
Epic customer instance / connected application | Each customer controls its independent instance and connections; there is no single central Epic endpoint. Public developer tools support registration and testing. Epic developer resources. | Your organization’s connection approval, exact permitted operations and implementation/support agreement. |
Oracle Health Millennium / appointment API | FHIR R4 documents appointment search and creation. Search by date excludes proposed appointments; the alternative date-or-request-period search includes them. Search semantics, creation documentation. | Customer configuration, authorized booking and cancellation route, supported status transitions and a complete rescheduling transaction. |
athenaOne / exchange and exit | athenahealth documents APIs, HL7/C-CDA interfaces and self-service machine-readable EHI export covering demographics, healthcare, billing and other datasets. Developer and export resources. | Exact interface access, export contents needed for your episode, fees and successful independent reconstruction. |
Vero / reviewed note transfer | Vero documents recording, typed context or uploads, template selection, note review and copy/export to PDF, Word, print or email. Note workflow. | A native EHR connection is not established by copy/export. Verify identity, receiving field and final saved version; this workspace does not satisfy scheduling requirements. |
“Supports FHIR” is not a complete interface specification
The HL7 FHIR R4 CapabilityStatement describes implementation capabilities, including supported resource interactions. Use it alongside the provider’s operation documentation and site-specific agreement. Record the resource, version, read or write operation, authorization scope, response, error handling and destination state. A documented search does not imply a documented write, and creating a booking does not establish an atomic rescheduling process.
For a concrete example, an appointment query that excludes proposed requests can make a queue look smaller than it is. Test the exact query and statuses your workflow needs. Similarly, a machine-readable export may be useful without containing every association or task your exit requires. Our EHR interoperability guide develops the technical exchange questions; the matrix here ties those answers to an operational acceptance decision.
Security and procurement evidence
Build a data-flow register that includes primary storage, cached copies, backups, logs, support access and subprocessors. Then connect each approval to a configured control and a test. The HHS risk-analysis guidance addresses assessment of risks and vulnerabilities to ePHI confidentiality, integrity and availability. A vendor questionnaire is input to that work, not its replacement.
For each supplier, obtain answers to six concrete questions:
- Who can enter? Identify workforce, service and support accounts; test sign-in, role changes and revocation. Include privileged and remote support access.
- What can they do? Specify allowed records and operations. Test denied writes directly, not merely whether the UI hides a control.
- Where does information go? List processing locations, support routes, subprocessors and secondary uses. Map permitted data to the contract.
- What evidence survives? Establish logged actor, operation, record association, time and result, with access and retention appropriate to the organization.
- How is failure handled? Name detection, notification, backup, recovery, reconciliation and the joint support owner when two products disagree.
- What changes after signing? Agree security-update support, end-of-life notice, interface changes, incident communication and exit assistance, with responsibilities and fees.
NIST SP 1305 provides a supplier-risk framework for establishing and communicating cybersecurity requirements. Use these procurement prompts to request evidence matching the actual entity, product, hosting region and period.
United States and Canada require separate legal mapping
For U.S. HIPAA-covered relationships, HHS cloud guidance explains that a cloud provider handling ePHI on a covered organization’s behalf can be a business associate even if it lacks the decryption key. Assess the applicable agreement and organizational responsibilities; encryption alone does not settle the relationship.
In Canada, map the applicable federal, provincial or territorial law and the parties’ actual roles. The OPC’s PIPEDA accountability bulletin addresses continued accountability for information transferred to processors, contractual protection and oversight. Include the relevant health-sector requirements in the same register.
For Ontario, the IPC’s 2021 Digital Health under PHIPA overview covers digital records, interoperability, audit logs and service-provider issues. The privacy lead should pair it with current law and the deployment’s roles. Record the source, jurisdiction, local owner, required control and retained evidence alongside hosting and contract details.
Access includes usability
Require representative users to perform the rescheduling and recovery tasks with their intended devices and assistive technologies. Use WCAG 2.2 for applicable web-accessibility requirements, then test keyboard access, readable errors and status announcements in the actual workflow. A conformance statement about the portal does not establish accessibility of a separate booking tool or staff recovery screen.
Compare the same operating scope and exit
Ask healthcare software providers to quote the same sites, roles, modules, volume, environments, interfaces and support hours. Separate license charges from configuration, mapping, connector hosting, validation, training, internal administration, exception handling and transition work.
Using fictional U.S.-dollar planning inputs, an interface cost model includes a $400 monthly connector, $6,000 setup allocated across 24 months and three internal support hours monthly at $60 per hour. That is $400 + $250 + $180 = $830 per month, before application subscriptions and other excluded work. Keep the upfront $6,000 payment visible; allocation changes the comparison period, not the invoice.
The model is useful only if support hours include actual exceptions. Count registration, scheduling, technical and records-team effort separately. Ask who pays when the portal vendor changes a field or the scheduler changes a status code. Released time is capacity unless a real budget reduction or useful reassignment is demonstrated.
Test exit before relying on it
For HS-08, the independent reader should reconstruct both appointment records, identifiers, effective and processing times, change history and the open communication task. If a portal export contains only the final booked appointment, record the missing cancelled entry and task explicitly. Decide whether another approved dataset supplies them; do not silently substitute a patient-readable PDF for the migration requirement.
Agree export format, scope, delivery time, fees, retained access, transition support and deletion responsibilities. Keep the original environment available until the responsible team accepts the reconstruction and retention plan. The medical records software guide explains the distinction between readable patient copies and reconstructable records in more detail.
Accept the workflow and retest changes
Make the decision with the people who own the affected domains. Technical reviewers approve interface behavior and recovery evidence; operations reviewers approve exception ownership and workload; clinical reviewers approve any clinical workflow implications. Record privacy and security decisions separately.
Retain one decision record with the scope, configurations, passed and failed cases, unrun checks, justified exclusions, defects, owners, evidence references and accountable approval. Review blocking failures individually, including wrong-patient writes and unauthorized operations. Assess each application against its assigned role in the complete workflow.
After implementation, monitor destination discrepancies, duplicate operations, oldest unresolved exceptions, communication failures and recovery effort. Include abandoned attempts and manual work rather than counting only successful API calls. Retest affected cases after a mapping, version, role, identity-provider or workflow change. A new feature can be useful and still reopen a previously settled system boundary.
The result should be a defensible choice: accept the bounded workflow, remediate a specific gap or decline the proposed configuration. That decision rests on traceable evidence and accountable ownership, not a vendor’s feature count.
Sources and further reading
- ONC 2025 SAFER Guides
- HL7 FHIR R4 CapabilityStatement
- Epic developer resources and customer connection model
- Oracle Health Millennium appointment search and appointment creation
- athenahealth interface and EHI export resources
- Vero note creation and export instructions
- HHS risk-analysis guidance and HIPAA cloud guidance
- OPC PIPEDA accountability bulletin
- IPC Ontario digital-health overview, 2021
- NIST SP 1305 supplier-risk guide
- W3C WCAG 2.2
Healthcare software systems questions
Practical answers about system boundaries, procurement and acceptance evidence.
What are healthcare software systems?
They are applications and services that support care, records, patient access, administration and operations. A healthcare system landscape usually includes several connected products, with different owners and authoritative records. Evaluate both each application and the handoffs between them.
How are healthcare software providers different from healthcare providers?
Healthcare software providers supply technology or related services; healthcare providers deliver care. Specify the supplier’s legal entity, product, module, hosting and integration responsibilities. A supplier relationship does not transfer the clinic’s care, privacy or operational accountability.
Should every application use the same patient identifier?
Not necessarily. Systems may have different local identifiers, but the approved crosswalk must preserve the correct association. Test mismatches and merges. A matching display name alone does not establish that two records belong to the same person.
Does a successful API response mean the workflow is finished?
No. It can acknowledge receipt or a technical operation without proving the required downstream state. Read back the appointment, record or task and verify outstanding work. Define transport acceptance and business completion separately.
Does FHIR support guarantee appointment write access?
No. Verify the exact resource, interaction, version, authorization and customer environment. Use the FHIR capability documentation to identify supported operations, then confirm licensing and demonstrate rescheduling at your site.
Can single sign-on replace application permission testing?
No. Authentication and authorization answer different questions. Test which records and operations each role can access, including direct API requests and service accounts. A user who can sign in must not automatically gain permission to edit every record.
Show all questions
What belongs in a requirements matrix?
Include a stable ID, boundary, required state, owner, controlled test and evidence reference. Add the supplier response, applicability, defect and approval in the internal procurement record. Keep the requirement linked to a user need rather than a marketing feature.
When is Not applicable appropriate?
Use it for a genuinely excluded function with an agreed reason. It is not a replacement for Not run or Fail. A drafting application may not own scheduling, but any scheduling workflow you are buying still needs an accountable owner and tested delivery path.
How should an organization test duplicate events?
Replay the same controlled event after an uncertain response and inspect destination counts and state. Retain the event identifier and retry evidence. Confirm that the receiving system applies the intended operation once and preserves any outstanding work.
Is a readable PDF enough for an exit test?
Usually not for a multi-system migration. Establish which structured records, associations, attachments, history and outstanding tasks must move, then reconstruct a synthetic episode independently. A patient-readable copy and a usable migration dataset solve different problems.
Does Canadian hosting establish privacy compliance?
No. Assess applicable federal, provincial or territorial law, permitted processing, support access, subcontractors and security controls. Record hosting location alongside those requirements in the data-flow and contract review.
What costs can appear outside the subscription?
Interfaces, implementation, data mapping, test environments, internal review, exception handling, release retesting and exit assistance can be separate costs. Compare the same sites, users, volume and support hours, and identify who pays when another supplier changes an interface.
How should AI modules be evaluated within healthcare systems?
Assess their intended task and permitted actions separately from connectivity. A generated note is not an executed order or completed appointment. Test source access, reviewer correction, receiving-record state and changes after model or configuration updates.
Who should approve system acceptance?
Use reviewers responsible for the affected clinical workflow, operations, records, privacy, security and integration domains. Record each domain’s decision, name the final decision authority and assign unresolved issues.
When should system-boundary tests be repeated?
Repeat affected cases after product, interface, identity-provider, mapping, permission or workflow changes. Retain stable synthetic fixtures for comparison. Monitor exceptions between releases; a passed pre-launch test is not proof that a later configuration behaves identically.
