Sponsor and owners
Executive sponsor, workspace owner, assessment lead, and named change owners.
Enterprise workshop guide
Azure DevOps Continuous Improvement Workshop
Establish the current state, agree the outcomes that matter, design the best feasible target, and leave with a measurable 30/60/90-day plan.
Assessment signal9 domains
Governance84
Identity69
Delivery76
Security58
Operations64
Decision dashboard
Current assessment state
Every number below is calculated from saved artifacts; blank records produce zero, not inferred conclusions.
0evidence records
0Critical or High findings
0/9supported maturity domains
0%action traceability
1630-minute modules
33dataset categories assessed
9maturity domains
430/60/90/Beyond roadmap waves
Operating sequence
One evidence thread, seven deliberate phases
Do not jump from observations to solutions. Each phase produces the inputs needed by the next.
01
Discover
Scope, stakeholders, outcomes, constraints, access.
02
Collect
Interviews, inventories, policies, histories, measures.
03
Validate
Reconcile evidence, anomalies, exceptions, gaps.
04
Assess
Maturity, risk, strengths, causes, confidence.
05
Design
Desired state, ideal controls, tradeoffs.
06
Plan
Changes, owners, sequence, measures, gates.
07
Review
Governance cadence, closure evidence, benefits.
Traceability model
Every recommendation earns its place
Evidence
Finding
Gap or outcome
Owned change
Measured result
Before the workshop
Prepare the room, the access, and the decision rights.
The assessment starts only when scope, participants, evidence handling, and read access are explicit.
Engagement record
Name the assessment before scoring it
These fields identify the engagement and travel with every export.
Before you start
Required tools and extensions
Install these on the collection workstation. See the collection prerequisites runbook for full context.
-
1. Azure CLI
Base command-line tool. Reopen the terminal after installing, then confirm the version.
winget install -e --id Microsoft.AzureCLI az version -
2. Azure DevOps CLI extension
Adds the
az devops,az repos,az pipelines,az boards, andaz artifactscommand groups used for evidence collection.az extension add --name azure-devops --upgrade az extension show --name azure-devops --output table -
3. PowerShell 7 or later
Runs the
scripts/collect-ado-evidence.ps1collector. Confirm you are on PowerShell 7+.winget install -e --id Microsoft.PowerShell $PSVersionTable.PSVersion -
4. Microsoft Entra sign-in and API access
Authenticate to the tenant connected to the Azure DevOps organization with an approved, read-only identity. Interactive Entra sign-in is preferred to a PAT.
az login --tenant '<tenant-id>' az account show --output table
Readiness checklist
Progress is retained in this browser.
People in the room
Decision-capable participation
Record how each group took part. This module cannot complete until every group is attested, so each decision carries a verifiable record of who participated rather than an unverified assumption.
Delivery voices
Project administrators, engineering representatives, and product or delivery leaders.
Control partners
Security, compliance, platform operations, identity, finance, and regional stakeholders as needed.
Evidence handling
Minimum evidence contract
Identity
Stable evidence ID, domain, source, scope, UTC collection time, and collector.
Reproducibility
Method and version, filters, pagination behavior, completeness, and known limitations.
Protection
Restricted location, classification, access list, retention, and disposal date. Never collect secret values.
16-module learning path
Work in focused 30-minute increments.
Complete the full eight-hour sequence or schedule individual modules. Each module ends with evidence that can be reviewed or resumed later.
SME-guided module plan
Every module is 30 minutes. Breaks and offline evidence collection are scheduled separately.
Prompts
Questions that expose the operating reality
Owners
- Which assets and delivery paths are business-critical?
- Where is variation intentional?
- Which risks can the organization accept?
Practitioners
- Where do controls add delay without reducing risk?
- What gets copied instead of reused?
- Can a production change be traced end to end?
Operations
- Which failures depend on one person?
- How are secrets and agents governed?
- What cannot be recovered or reported today?
Quantitative collection
Assess 33 data categories without turning absence into evidence.
The starter collector automates 25 inventory datasets. Use the runbook for the remaining API, CLI, analysis, and manual reconciliation steps; record permission gaps and partial results as limitations.
Evidence catalog
Record what was actually collected
Store metadata and restricted links here. Keep raw evidence outside this file.
Collection rules
Completeness is a claim to prove
Page every list
Follow the endpoint-specific continuation mechanism. Parameter names vary by service.
Reconcile the UI
Ownership, intent, billing, criticality, and some policy settings require manual attestation.
Protect the raw evidence
Hash exports, restrict access, redact summaries, and keep access tokens only in memory.
Assessment workspace
Score what is observed, then make uncertainty visible.
Maturity is not risk. A low-maturity capability may be acceptable; a narrow high-impact exposure may be urgent.
Finding register
Make every conclusion earn its evidence
A finding requires valid evidence or an explicit approved limitation.
Domain maturity
0 Unknown, 1 Ad hoc, 2 Repeatable, 3 Defined, 4 Measured, 5 Optimizing.
| Domain | Current | Target | Gap | Evidence | Confidence | Rationale |
|---|
Finding risk
Likelihood x impact
Risk score12High
Response playbook
Turn evidence into a governed decision
Tool severity informs risk; context, ownership, and refreshed evidence determine disposition and closure.
1. Validate
Confirm scope, owner, timestamp, detection logic, and coverage. Reproduce or reconcile with a second source.
2. Contextualize
Consider exploitability, privilege, production reach, criticality, existing controls, and obligations.
3. Disposition
Remediate, contain, accept through an exception, report a demonstrated inaccuracy, or close as no longer applicable.
4. Own
Assign one owner, due date, acceptance evidence, success measure, dependencies, and residual-risk approver.
5. Verify
Independently test the control, refresh originating evidence, confirm residual risk, and monitor recurrence.
Finding quality
Ready for owner review when...
Target design
Reconcile owner intent with the best feasible enterprise state.
The ideal state is neither today's preference nor maximum maturity everywhere. It is the approved intersection of outcomes, guidance, risk, cost, and capacity.
By [date], the Azure DevOps organization will enable
[teams and personas] to achieve
[outcomes] while meeting [constraints].
Success will be demonstrated by [measures and targets].
The organization intentionally accepts
[tradeoffs or exceptions].
Enterprise baseline
Control globally, delegate deliberately
Topology and geography
Start simple; split only for justified hard boundaries, residency, administration, identity tenant, or extreme scale.
Governance and identity
Entra-backed groups, least privilege, PIM, Conditional Access, lifecycle automation, audit streaming, and short-lived authentication.
Boards and reporting
Durable product teams, common metric semantics, portfolio visibility, and governed Analytics or Power BI reporting.
Repositories
Named ownership, protected branches, constrained bypass, security scanning, and reversible lifecycle management.
Pipelines and releases
YAML, secured templates, project-scoped identities, protected resources, federated service connections, and isolated agents.
Operations
Owned integrations, service-protection-aware automation, limit monitoring, support paths, continuity exercises, and recurring review.
Design decisions
Record the tradeoffs
Desired outcomes
Define measurable success
Outcomes anchor gaps and changes to an owner, measure, target, and review date.
Gap register
Connect findings to outcomes
Every gap must trace to at least one evidence-backed finding or measurable outcome.
Change planning
Sequence risk reduction, foundations, and scale.
Every accepted change needs one owner, traceable rationale, dependencies, effort, acceptance evidence, and a success measure.
Prioritization
Score a candidate change
Priority score152(Risk) + Value + Urgency + Enablement - Effort
Change catalog
Own the remediation
Changes are accepted only when their rationale, owner, acceptance evidence, measure, dependencies, and wave are explicit.
Roadmap
Owned waves with explicit exit evidence
Completion and handoff
Close the workshop only when the operating loop can continue.
A workshop is complete when decisions, owners, evidence, measures, and review dates survive the meeting.
Completion gates
Use the final state as the sign-off agenda.
Exception register
Govern every accepted deviation
Exceptions require finding traceability, a compensating control, accountable owner, and expiry.
Decision record
Approve work or expose disagreement
This local decision record is not identity verified. It must be reconciled with an approved record before reliance.
Assessment sign-off
Commit the accountable roles
Local sign-off records acceptance or conditions but is not identity verified; checklist ticks do not substitute for named decisions.
Governance cadence
Keep the plan alive
Weekly
Remediation stand-up through day 90: progress, blockers, dependency actions.
Biweekly or monthly
Risk and exception review: acceptance, expiration, containment, escalation.
Monthly
Review privileged and newly consented applications, stale credentials, high-risk alerts, and overdue remediation.
Monthly and quarterly
Platform governance and sponsor checkpoints: standards, adoption, outcomes, capacity, roadmap changes.
Quarterly and on change
Reattest in-scope applications and service identities; repeat after owner, permission, credential, or usage change.
Source set
Workshop documents and Microsoft guidance
Collection runbookExact API, CLI, manual, metric, and closeout
procedures.
Workshop charterAssessment method, worksheets, target design, roadmap, and
governance.
Enterprise structureOrganizations, projects, teams, repositories, and
business-unit mapping.
Azure DevOps securityOfficial security configuration and operating
guidance.
Secure Azure PipelinesProject isolation, YAML, identities, resources, agents, and
secrets.
Integration best practicesBatching, reporting APIs, query efficiency, and service
protection.
Authentication guidanceEntra tokens, managed identities, service principals, and
PAT selection.
Application permissionsReview and revoke delegated and application permissions in
Microsoft Entra.
Advanced SecurityOrganization risk, coverage, code, dependency, and secret
alerts.
Vulnerability managementRisk-based discovery, prioritization, remediation, and
exception handling.