[Docutils-develop] 1.0.0 & API

Would it help to adopt semantic versioning (http://semver.org)?

...

We do this: `rst2html --version` --> 0.13.2a
Major version zero (0.y.z) is for initial development. Anything may
change at any time. The public API should not be considered stable.
-- http://semver.org/

If your software is being used in production, it should probably
already be 1.0.0. If you have a stable API on which users have come
to depend, you should be 1.0.0. If you’re worrying a lot about
backwards compatibility, you should probably already be 1.0.0.
All of these apply to docutils.

Agreed. However,
For this system to work, you first need to declare a public API. This
may consist of documentation or be enforced by the code itself.
Regardless, it is important that this API be clear and precise.
What belongs to the *public API* of Docutils?
* behaviour specified in the docs
* behaviour specified in the tests
- modules
- classes
- arguments, expected type/values
- methods
- functions
- arguments and return values of functions and methods
- side-effects of functions and methods
(e.g. handling of the "context" stack in the HTML writer)
?????

All of the above. Unless something is explicitly marked "private" or
"internal only", it's part of the public API.

As for the HTML writer's context stack, the rules for it are part of
the public API (i.e. matching pushes & pops in methods). If client
code doesn't get it right, that's not our problem.

Should the next release be 1.0.0 then?
Probably.
(½ ;-)

We should consider some clean-up first, e.g.
* remove the `handle_io_errors` option from io.FileInput/Output.
.. used by Sphinx up to version 1.3.1,
see https://github.com/sphinx-doc/sphinx/issues/1834
- "doctest" element -> literal block with "pycon" (python-console)
class argument and syntax highlight (like the "code" directive),
- special admonitions (note, hint, warning, ...) -> generic "admonition"
element with class attribute and auto-generated title.
(see RELEASE-NOTES).

Was there ever any discussion of these items? I can only find their
SVN commit messages.

What is the justification for these removals?

Without justification: If it ain't broke, don't fix it.

The text on Docutils' home page (http://docutils.sourceforge.net/),
Docutils is in active development and APIs should be considered
experimental subject to change at any time. Although any time is a
time after serious thought and discussion.
was written so long ago that I don't remember when. Initial
development was many years ago; lately it's incremental features and
bug fixes. We've been hiding behind the 0 in 0.y.z for a long time —
too long. It's a bit embarrassing at this point.

Yes, development was so slow and cautious for most of the last years, that
our "customers" got used to a stable code base and the above warning got
mostly forgotten.

And I don't recall much serious discussion (second part of the quote
above) about this particular API change.

There was discussion about the behaviour change and the change to rst
syntax, not about the "accidential" API change.

If this API existed in a previous released version, the incompatible
change should not have happened.

Yes, if all function and method arguments and return values are
considered part of the "code-defined" API, then this should not have
happened.

There are ways of adding to an API while maintaining
backwards compatibility, and Docutils should be using them. Many
people and a lot of software out there depend on Docutils.
Perhaps we should release a 0.13.2 or 0.14 which fixes the
incompatibility? (Followed by 1.0.0?) Of course, we'd need a volunteer
to actually implement the fix.

Try the patch at
https://sourceforge.net/p/docutils/bugs/_discuss/thread/48306ff1/d700/attachment/colwidth-auto-safe.patch

Now that the 0.13.1 version of those methods is published, wouldn't
reverting to the previous version potentially break code that has
adapted? If I were writing client code and this happened, I'd be
really frustrated and angry.

I suggest that first we get the opinion of Dmitry Shachnev who
reported the issue, so that this "fix" doesn't come back to bite him.
If he's OK with it, we should restore the old API with a bugfix
release, i.e. 0.13.2.

Or do we just chalk it up to experience
and try to do better next time?

This would be possible, too.

If we can fix it, we should.

DG

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Repl

Guenter Milde

2016-12-23 19:03:13 UTC

What belongs to the *public API* of Docutils?
* behaviour specified in the docs
* behaviour specified in the tests
- modules
- classes
- arguments, expected type/values
- methods
- functions
- arguments and return values of functions and methods
- side-effects of functions and methods
(e.g. handling of the "context" stack in the HTML writer)

All of the above. Unless something is explicitly marked "private" or
"internal only", it's part of the public API.

In this case, I suggest a review looking for auxiliary functions that we may
want to exclude from the API before release 1.x.

Post by David Goodger
As for the HTML writer's context stack, the rules for it are part of
the public API (i.e. matching pushes & pops in methods). If client
code doesn't get it right, that's not our problem.

The problem was a bit more tricky:

The Sphinx writer overwrites the parent visit_image() (which pushs one
item to `context` in 0.12 but not in 0.13). A conditional either calls the
parent method or an alternative that pushes one item.

The depart_image() function was, however, not called.

+ push and pop did match in 0.12
- push and pop did not match in 0.13

Is this our fault or theirs...?

More important: how should we handle such changes in the future?

- push empty item in case it is no longer required just to be sure,
- expect downstream users to always call both, _visit() and _depart()?

Should the next release be 1.0.0 then?
Probably.
(½ ;-)

Was there ever any discussion of these items? I can only find their
SVN commit messages.

The issues came up such a long time ago, I don't know how explicitly they
were discussed in the list. In any case, they are in todo.txt since
oct. 2009.

Post by David Goodger
What is the justification for these removals?

Make the document model simpler and more consistent:

Doctree pruning
---------------

The number of doctree nodes can be reduced by "normalizing" some related
nodes. This makes the document model and the writers somewhat simpler.

* The "doctest" element should go away. The construct could simply be
a front-end to generic literal blocks. We could immediately (in 0.7)
remove the doctest node from the doctree, but leave the
syntax in reST. The reST parser could represent doctest blocks as
literal blocks with a class attribute. The syntax could be left in
reST (for a set period of time?).

* "Normalize" special admonitions (note, hint, warning, ...) during parsing
(similar to _`transforms.writer_aux.Admonitions`). There is no need to
keep them as distinct elements in the doctree specification.

Keep the special admonition directives in reStructuredText syntax?

-- todo.txt

Post by David Goodger
Without justification: If it ain't broke, don't fix it.

+1 writers would not need special visit/depart methods for all admonition
types.

+1 doctest-blocks would become a special case of code and "naturally" gain
from the syntax highlight.

...

Perhaps we should release a 0.13.2 or 0.14 which fixes the
incompatibility? (Followed by 1.0.0?) Of course, we'd need a volunteer
to actually implement the fix.

Try the patch at
https://sourceforge.net/p/docutils/bugs/_discuss/thread/48306ff1/d700/attachment/colwidth-auto-safe.patch

Now that the 0.13.1 version of those methods is published, wouldn't
reverting to the previous version potentially break code that has
adapted? If I were writing client code and this happened, I'd be
really frustrated and angry.
I suggest that first we get the opinion of Dmitry Shachnev who
reported the issue, so that this "fix" doesn't come back to bite him.
If he's OK with it, we should restore the old API with a bugfix
release, i.e. 0.13.2.

OK.

I checked Dmitries patches, they are safe with both.)

Reverting the "context" stack use in html4css1 for Sphinx is not safe with
the original patch there.

Günter

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Repl

Brecht Machiels

2016-12-23 22:06:52 UTC

Hi,

I currently don't have much time to join in on the discussion, but I'll
chip with some general remarks.

Günther, it seems like you want to make some significant changes before
1.0.0 is released. I think this might take a very long time. However,
if you're serious about using semver, docutils should already be 1.0.0
since the API (however you define it) has been stable and widely used
for many years now. So I would suggest moving to 1.0.0 without further
changes, with the exception of a fix for the incompatibility introduced
by yours truly.

Once your list of suggested changes has been implemented, the version
number can jump to 2.0.0. Note that there is nothing special about
1.0.0; it is follows from the semver rules.

It would be good to more clearly define what defines the API exactly. I
would assume that all public functions/methods/attributes are part of
the API, for example. But does the docutils codebase differentiate
between private (underscore) and public symbols? Once 1.0.0 is
released, this definition will determine which changes go in 1.1.0 and
which go in 2.0.0.

Best regards,
Brecht

All of the above. Unless something is explicitly marked "private" or
"internal only", it's part of the public API.

In this case, I suggest a review looking for auxiliary functions that we may
want to exclude from the API before release 1.x.

The Sphinx writer overwrites the parent visit_image() (which pushs one
item to `context` in 0.12 but not in 0.13). A conditional either calls the
parent method or an alternative that pushes one item.
The depart_image() function was, however, not called.
+ push and pop did match in 0.12
- push and pop did not match in 0.13
Is this our fault or theirs...?
More important: how should we handle such changes in the future?
- push empty item in case it is no longer required just to be sure,
- expect downstream users to always call both, _visit() and _depart()?

Should the next release be 1.0.0 then?
Probably.
(½ ;-)

Was there ever any discussion of these items? I can only find their
SVN commit messages.

The issues came up such a long time ago, I don't know how explicitly they
were discussed in the list. In any case, they are in todo.txt since
oct. 2009.

Post by David Goodger
What is the justification for these removals?

Doctree pruning
---------------
The number of doctree nodes can be reduced by "normalizing" some related
nodes. This makes the document model and the writers somewhat simpler.
* The "doctest" element should go away. The construct could simply be
a front-end to generic literal blocks. We could immediately (in 0.7)
remove the doctest node from the doctree, but leave the
syntax in reST. The reST parser could represent doctest blocks as
literal blocks with a class attribute. The syntax could be left in
reST (for a set period of time?).
* "Normalize" special admonitions (note, hint, warning, ...) during parsing
(similar to _`transforms.writer_aux.Admonitions`). There is no need to
keep them as distinct elements in the doctree specification.
Keep the special admonition directives in reStructuredText syntax?
-- todo.txt

Post by David Goodger
Without justification: If it ain't broke, don't fix it.

+1 writers would not need special visit/depart methods for all admonition
types.
+1 doctest-blocks would become a special case of code and "naturally" gain
from the syntax highlight.
...

Perhaps we should release a 0.13.2 or 0.14 which fixes the
incompatibility? (Followed by 1.0.0?) Of course, we'd need a volunteer
to actually implement the fix.

Try the patch at
https://sourceforge.net/p/docutils/bugs/_discuss/thread/48306ff1/d700/attachment/colwidth-auto-safe.patch

Now that the 0.13.1 version of those methods is published, wouldn't
reverting to the previous version potentially break code that has
adapted? If I were writing client code and this happened, I'd be
really frustrated and angry.
I suggest that first we get the opinion of Dmitry Shachnev who
reported the issue, so that this "fix" doesn't come back to bite him.
If he's OK with it, we should restore the old API with a bugfix
release, i.e. 0.13.2.

OK.
I checked Dmitries patches, they are safe with both.)
Reverting the "context" stack use in html4css1 for Sphinx is not safe with
the original patch there.
Günter
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Docutils-develop mailing list
https://lists.sourceforge.net/lists/listinfo/docutils-develop
Please use "Reply All" to reply to the list.

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Guenter Milde

2016-12-27 13:26:53 UTC

Dear all,

I hope you had a nice Christmas.

Günter, it seems like you want to make some significant changes before
1.0.0 is released.

I see it rather as a cleanup, not significant changes.

I think this might take a very long time.

If the discussion remains at the speed we had with the html5 writer, yes.
But as David announced a return to active Docutils development, there is
hope for the better.

However, if you're serious about using semver, docutils should already
be 1.0.0 since the API (however you define it) has been stable and
widely used for many years now. So I would suggest moving to 1.0.0
without further changes, with the exception of a fix for the
incompatibility introduced by yours truly.
Once your list of suggested changes has been implemented, the version
number can jump to 2.0.0. Note that there is nothing special about
1.0.0; it is follows from the semver rules.

I agree that the "de facto" state would have allowed stepping to 1.x some
time ago. Still, a step in the "major" number for a maintenance release
just to "catch up" does not seem necessary to me. I propose to relase a
fixup 0.13.2 and prepare 1.0 with a clearly stated API and a fixed
docutils.dtd.

Günter

David Goodger

2017-01-02 22:00:19 UTC

Post by Guenter Milde
Dear all,
I hope you had a nice Christmas.

Günter, it seems like you want to make some significant changes before
1.0.0 is released.

I see it rather as a cleanup, not significant changes.

I think this might take a very long time.

If the discussion remains at the speed we had with the html5 writer, yes.
But as David announced a return to active Docutils development, there is
hope for the better.

But I'm going to fight against a lot of the proposed changes. So my
presence may not make the process any more efficient than my absence
;-)

Clearly stated API: What do you propose? Should there be a document
listing every module, class, function, and attributes thereof? Why
make so much work for ourselves?

Adding things is not an issue. Removing things is. So don't remove
things. We never know what client code may depend on what. Changing
function/method signatures is also an issue. So don't change these in
a backwards incompatible way.

DTD: -1. See my other message today.

As for the "cleanup": -1. Better to go 1.0 with what we have now than
to put it off, because that may be a *long* time coming. Better now
than never (or a long time from now), the perfect is the enemy of the
good, etc.

1.0.0 shall be after 0.13.2 though, because that should just be a
bugfix release.

The benefit of releasing 1.0.0 will be pure psychology & PR. It sends
a signal that the software is at a certain level of maturity, and our
commitment. Also it will hopefully allow us to avoid bugs like we're
seeing in 0.13.1.

David Goodger
<http://python.net/~goodger>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Alan G. Isaac

2017-01-02 22:39:37 UTC

In this bugfix release, might it at long last be time to fix bug 209?
https://sourceforge.net/p/docutils/bugs/209/

It's just a writer bug, but it is longstanding and very annoying.

Thank you,
Alan Isaac

Post by David Goodger
1.0.0 shall be after 0.13.2 though, because that should just be a
bugfix release.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

David Goodger

2017-01-02 23:30:29 UTC

Sure. We just need somebody to fix the issue and submit a patch.

Repeatedly complaining about it doesn't add anything new and is in
itself very annoying. If you can't fix it, and can't get anybody who
could fix it to be interested, don't expect anything to change. (I
don't use LaTeX, cannot fix the bug, and am not interested.)

Also, hijacking unrelated threads is very annoying. Please stop doing
that. Start a new thread with an appropriate subject, and reply
*there* instead.

David Goodger
<http://python.net/~goodger>

Post by Alan G. Isaac
In this bugfix release, might it at long last be time to fix bug 209?
https://sourceforge.net/p/docutils/bugs/209/
It's just a writer bug, but it is longstanding and very annoying.
Thank you,
Alan Isaac

Post by David Goodger
1.0.0 shall be after 0.13.2 though, because that should just be a
bugfix release.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
https://lists.sourceforge.net/lists/listinfo/docutils-develop
Please use "Reply All" to reply to the list.

Alan G. Isaac

2017-01-02 23:45:48 UTC

Was there ever developer approval of my proposed solution,
included with the bug report?
https://sourceforge.net/p/docutils/bugs/209/
I don't have a record of that.

Thank you,
Alan Isaac

Post by David Goodger
Repeatedly complaining about it doesn't add anything new and is in
itself very annoying.

David Goodger

2017-01-03 00:20:22 UTC

I don't recall. There's nothing on the bug report, which there would
be. ISTR some discussion a long time ago. I suspect the other
developers (such as the one(s) who can fix this) either don't care,
aren't bothered by the issue, or have been put off by your badgering.

And proposing a solution is nothing like actually delivering a patch.
It's like telling your car mechanic, "I think it's the transmission.
You should check the transmission. Did you check the transmission?" If
you know how to fix it, just fix it yourself.

David Goodger
<http://python.net/~goodger>

Post by Alan G. Isaac
Was there ever developer approval of my proposed solution,
included with the bug report?
https://sourceforge.net/p/docutils/bugs/209/
I don't have a record of that.
Thank you,
Alan Isaac

Post by David Goodger
Repeatedly complaining about it doesn't add anything new and is in
itself very annoying.

Alan G. Isaac

2017-01-03 02:20:05 UTC

I do not understand why your tone is so aggressive.

There is no point in developing a patch unless
there is some chance of it being accepted. If the
developers state that wrapping literal blocks in a
custom environment is an acceptable solution, then
I have an incentive to explore writing a patch, even
though I know nothing about docutils internals.

Alan

Post by David Goodger
I suspect the other
developers (such as the one(s) who can fix this) either don't care,
aren't bothered by the issue, or have been put off by your badgering.
And proposing a solution is nothing like actually delivering a patch.

Ben Finney

2017-01-03 03:13:44 UTC

Post by Alan G. Isaac
I do not understand why your tone is so aggressive.

I expect both parties feel the other is being agressive. That's because
you both have expectations that are not compatible, and the other is
violating them.

Alan, your expectation is that reporting an issue and proposing what
needs to be done, is sufficient, and that the only thing which needs to
be done to resolve the issue is to remind the project that it needs
attention.

David, on the other hand, has the expectation that progress on the issue
can only be made by some interested party producing an implementation in
code, that is verified to fix the problem, in such a way that the
project can accept it into the code base.

So, when Alan reminds the project that the issue is still open, this is
seen by David as unhelpful nagging without actually contributing to the
resolution of the issue.

And when David points out that no-one needs reminding of the open issue
and that doing so repeatedly is nagging, that is seen by Alan as
hostility.

As someone not invested in either side, I must say that I agree with
David's assessment. David is not being hostile; he's pointing out
reality, and asking firmly that the issue should only be discussed in
the context of *new information*: something that changes what to do
about the issue, or progress on an actual implementation to resolve it.

Failing that, Alan, you'll just have to find other outlets. Find someone
to be interested enough to do the work (maybe yourself), or be patient
until that happens.
--
\ “I must say that I find television very educational. The minute |
`\ somebody turns it on, I go to the library and read a book.” |
_o__) —Groucho Marx |
Ben Finney

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply

David Goodger

2017-01-03 03:17:37 UTC

Post by Ben Finney

Post by Alan G. Isaac
I do not understand why your tone is so aggressive.

I expect both parties feel the other is being agressive. That's because
you both have expectations that are not compatible, and the other is
violating them.
Alan, your expectation is that reporting an issue and proposing what
needs to be done, is sufficient, and that the only thing which needs to
be done to resolve the issue is to remind the project that it needs
attention.
David, on the other hand, has the expectation that progress on the issue
can only be made by some interested party producing an implementation in
code, that is verified to fix the problem, in such a way that the
project can accept it into the code base.
So, when Alan reminds the project that the issue is still open, this is
seen by David as unhelpful nagging without actually contributing to the
resolution of the issue.
And when David points out that no-one needs reminding of the open issue
and that doing so repeatedly is nagging, that is seen by Alan as
hostility.
As someone not invested in either side, I must say that I agree with
David's assessment. David is not being hostile; he's pointing out
reality, and asking firmly that the issue should only be discussed in
the context of *new information*: something that changes what to do
about the issue, or progress on an actual implementation to resolve it.
Failing that, Alan, you'll just have to find other outlets. Find someone
to be interested enough to do the work (maybe yourself), or be patient
until that happens.

Thank you, very well said!

David Goodger

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

David Goodger

2017-01-03 03:14:20 UTC

Post by Alan G. Isaac
I do not understand why your tone is so aggressive.

I wouldn't say "aggressive". Maybe "hostile".
;-)

I just searched through my email, and found several instances of you
asking the same question [*]_. Your question has been explicitly
answered before. I suspect that Engelbert and Günter are just tired of
hearing about it from you. Asked and answered sir!

.. [*] BTW: this is about literal blocks in a quote environment. Never
fully mentioned in this thread so far, so *this thread* didn't show up
in my search! And the subject, "bug 209", means nothing to me or
anyone else: thread subject fail.

Post by Alan G. Isaac
There is no point in developing a patch unless
there is some chance of it being accepted. If the
developers state that wrapping literal blocks in a
custom environment is an acceptable solution, then
I have an incentive to explore writing a patch, even
though I know nothing about docutils internals.

How have periodic reminders, sometimes via hijacked threads, worked out so far?

"Insanity: doing the same thing over and over again and expecting
different results." — Albert Einstein

I suspect that if a patch does appear, that solves your problem and
doesn't break anything else (including tests!), it will be easy to
apply. I'd push for it, if only to be rid of this issue.

LaTeX gurus: please weigh in on this
(https://sourceforge.net/p/docutils/bugs/209/). Is there a chance of a
patch being accepted? If not, please reject it with an explanation
why. Search email for prior explanations to copy&paste.

David Goodger
<http://python.net/~goodger>

Post by Alan G. Isaac

Alan G. Isaac

2017-01-03 12:48:04 UTC

Post by David Goodger
I suspect that if a patch does appear, that solves your
problem and doesn't break anything else (including
tests!), it will be easy to apply. I'd push for it, if
only to be rid
of this issue.
LaTeX gurus: please weigh in on this
(https://sourceforge.net/p/docutils/bugs/209/). Is there
a chance of a patch being accepted? If not, please reject
it with an explanation why.

Thank you. Just to elaborate, is there interest in a patch that
would wrap literal blocks in a custom environment instead of in
a ``quote`` environment? (From prior discussion, that is the only
solution I was able to see, since iirc there was i. commitment to
the ``alltt`` environment and ii. commitment to a default behavior
of indenting this environment.) As an explanation for those not used to
LaTeX: one cannot separately style an environment based on child
or parent environments.

So for example, instead of the current LaTeX output like
this::

\begin{quote}
Explicit is better than implicit.
\end{quote}

\begin{quote}
\begin{alltt}
def myfunc():
pass
\end{alltt}
\end{quote}

The latter (literal) environment would look like::

\begin{docutilsIndentLiteral}
\begin{alltt}
def myfunc():
pass
\end{alltt}
\end{docutilsIndentLiteral}

where the ``docutilsIndentLiteral`` environment (or whatever
name is chosen), identical to the definition of ``quote`` is
provided in the document header::

\newenvironment{docutilsIndentLiteral}
{\list{}{\rightmargin\leftmargin}%
\item\relax}
{\endlist}

This leaves the current behavior unchanged but allows users
to separately alter their quote and literal block styles.

Thank you,
Alan Isaac

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Guenter Milde

2017-01-10 20:37:38 UTC

Post by Guenter Milde
If the discussion remains at the speed we had with the html5 writer, yes.
But as David announced a return to active Docutils development, there is
hope for the better.

But I'm going to fight against a lot of the proposed changes. So my
presence may not make the process any more efficient than my absence
;-)

Clearly stated API: What do you propose? Should there be a document
listing every module, class, function, and attributes thereof?

No.

But, e.g., a statement how to avoid the problem of the last Sphinx
incompatibility:

A project uses Docutils as library,
subclasses writers.html4css1.HTMLTranslator(),
overwrites a visit_*() method (that pushes one item to the context stack),
does not overwrite the matching depart_*() method.

Docutils changes the visit/depart method pair not to use the context stack.

The problem can be avoided by:

* dummy push/pop of the context stack in the new Docutils version
(stable API also for side-effects), or

* defensive programming in the "client" program:
+ always overwrite both visit/depart
+ if calling the parent visit_*(), also call the parent depart_*().

Post by David Goodger
Adding things is not an issue. Removing things is. So don't remove
things. We never know what client code may depend on what. Changing
function/method signatures is also an issue. So don't change these in
a backwards incompatible way.

Also, clear rules on how to implement backwards incompatible changes
(announce time, pre-release ...) would be good.

Post by David Goodger
As for the "cleanup": -1. Better to go 1.0 with what we have now than
to put it off, because that may be a *long* time coming. Better now
than never (or a long time from now), the perfect is the enemy of the
good, etc.
1.0.0 shall be after 0.13.2 though, because that should just be a
bugfix release.
The benefit of releasing 1.0.0 will be pure psychology & PR. It sends
a signal that the software is at a certain level of maturity, and our
commitment.

And it removes the contradiction that version number and documentation
say: "the API is experimental" but actual API development stalled long
ago and users expect a stable API by now.

Günter

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to

David Goodger

2017-01-16 22:38:34 UTC

Post by Guenter Milde
If the discussion remains at the speed we had with the html5 writer, yes.
But as David announced a return to active Docutils development, there is
hope for the better.

But I'm going to fight against a lot of the proposed changes. So my
presence may not make the process any more efficient than my absence
;-)

Clearly stated API: What do you propose? Should there be a document
listing every module, class, function, and attributes thereof?

No.
But, e.g., a statement how to avoid the problem of the last Sphinx
A project uses Docutils as library,
subclasses writers.html4css1.HTMLTranslator(),
overwrites a visit_*() method (that pushes one item to the context stack),
does not overwrite the matching depart_*() method.
Docutils changes the visit/depart method pair not to use the context stack.
* dummy push/pop of the context stack in the new Docutils version
(stable API also for side-effects), or
+ always overwrite both visit/depart
+ if calling the parent visit_*(), also call the parent depart_*().

The latter option should definitely be part of the module
documentation (e.g. a "how to use this module" section in the module
docstring).

The former certainly wouldn't hurt much, only adding cruft in the long
run. But what about when we change a visit/depart method pair to use
the context stack, when formerly they didn't? There's no way to keep
"side effect API" backwards compatibility in this case. So I think we
should not guarantee such backwards compatibility.

Also, clear rules on how to implement backwards incompatible changes
(announce time, pre-release ...) would be good.

Adding and announcing a release candidate (eventually maybe alpha &
beta releases too) before the final release would be useful too. Maybe
that's what you meant by pre-release.

And it removes the contradiction that version number and documentation
say: "the API is experimental" but actual API development stalled long
ago and users expect a stable API by now.

Yes. That's partly what I meant by psychology & PR.

David Goodger
<http://python.net/~goodger>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Guenter Milde

2017-01-17 22:56:37 UTC

...

Post by David Goodger
Clearly stated API: What do you propose? Should there be a document
listing every module, class, function, and attributes thereof?

No. But, e.g., a statement how to avoid the problem of the last Sphinx
A project uses Docutils as library,
subclasses writers.html4css1.HTMLTranslator(),
overwrites a visit_*() method (that pushes one item to the context stack),
does not overwrite the matching depart_*() method.
Docutils changes the visit/depart method pair not to use the context stack.
* dummy push/pop of the context stack in the new Docutils version
(stable API also for side-effects), or
+ always overwrite both visit/depart
+ if calling the parent visit_*(), also call the parent depart_*().

The latter option should definitely be part of the module documentation
(e.g. a "how to use this module" section in the module docstring).

How about amending the HTMLTranslator class docstring instead of

"""When subclassing HTMLTranslator, ..."""
?

The former certainly wouldn't hurt much, only adding cruft in the long
run. But what about when we change a visit/depart method pair to use
the context stack, when formerly they didn't? There's no way to keep
"side effect API" backwards compatibility in this case. So I think we
should not guarantee such backwards compatibility.

Agreed. Where could we document this?

...

Also, clear rules on how to implement backwards incompatible changes
(announce time, pre-release ...) would be good.

Adding and announcing a release candidate (eventually maybe alpha &
beta releases too) before the final release would be useful too. Maybe
that's what you meant by pre-release.

Yes.

Thanks,
Günter

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

P

David Goodger

2017-01-17 23:28:49 UTC

Post by David Goodger
Clearly stated API: What do you propose? Should there be a document
listing every module, class, function, and attributes thereof?

No. But, e.g., a statement how to avoid the problem of the last Sphinx
A project uses Docutils as library,
subclasses writers.html4css1.HTMLTranslator(),
overwrites a visit_*() method (that pushes one item to the context stack),
does not overwrite the matching depart_*() method.
Docutils changes the visit/depart method pair not to use the context stack.
* dummy push/pop of the context stack in the new Docutils version
(stable API also for side-effects), or
+ always overwrite both visit/depart
+ if calling the parent visit_*(), also call the parent depart_*().

The latter option should definitely be part of the module documentation
(e.g. a "how to use this module" section in the module docstring).

How about amending the HTMLTranslator class docstring instead of
"""When subclassing HTMLTranslator, ..."""
?

Sure. Specifically, it should be documented in
docutils.writers._html_base.HTMLTranslator. Maybe subclasses
(docutils.writers.html4css1.HTMLTranslator etc.) should explicitly
refer the reader to the superclass: ".. IMPORTANT:: see the docstring
of docutils.writers._html_base.HTMLTranslator for usage instructions".

Hmm... docutils.writers.pep_html.HTMLTranslator defines
depart_field_list but not visit_field_list. Should we explicitly
define it as the following (for internal consistency
#eatourowndogfood)? ::

def visit_field_list(self, node):
# doesn't use context stack:
html4css1.HTMLTranslator.visit_field_list(self, node)

Or maybe even::

def visit_field_list(self, node):
# ensure that we're not using the context stack:
context_stack_length = len(self.context)
html4css1.HTMLTranslator.visit_field_list(self, node)
assert len(self.context) == context_stack_length

Other subclasses should be reviewed.

Agreed. Where could we document this?

Same place: HTMLTranslator class docstring.

To be clear, I mean that we don't guarantee any "side effect API"
across versions. In this case, specifically, we may change our use of
the context stack across versions, adding or removing stack usage.
Subclasses must use the context stack consistently in visit/depart
pairs.

David Goodger
<http://python.net/~goodger>

Also, clear rules on how to implement backwards incompatible changes
(announce time, pre-release ...) would be good.

Adding and announcing a release candidate (eventually maybe alpha &
beta releases too) before the final release would be useful too. Maybe
that's what you meant by pre-release.

Yes.
Thanks,
Günter

Guenter Milde

2017-01-18 10:23:00 UTC

...

Post by Guenter Milde
a statement how to avoid the problem of the last Sphinx
A project uses Docutils as library,
subclasses writers.html4css1.HTMLTranslator(),
overwrites a visit_*() method (that pushes one item to the context stack),
does not overwrite the matching depart_*() method.
Docutils changes the visit/depart method pair not to use the context stack.
* dummy push/pop of the context stack in the new Docutils version
(stable API also for side-effects), or
+ always overwrite both visit/depart
+ if calling the parent visit_*(), also call the parent depart_*().

The latter option should definitely be part of the module documentation
(e.g. a "how to use this module" section in the module docstring).

How about amending the HTMLTranslator class docstring instead ...

...

Post by David Goodger
Sure. Specifically, it should be documented in
docutils.writers._html_base.HTMLTranslator. Maybe subclasses
(docutils.writers.html4css1.HTMLTranslator etc.) should explicitly
refer the reader to the superclass: ".. IMPORTANT:: see the docstring
of docutils.writers._html_base.HTMLTranslator for usage instructions".

Something like ::

class HTMLTranslator(nodes.NodeVisitor):
"""Generic Docutils to HTML translator.

See the html4css1 and html5_polyglot for writers for full featured HTML
writers.

Safe use:
the `visit_*` and `depart_*` methods use a the
heterogeneous stack `self.context`.

When subclassing, make sure to be consistent:

+ Always overwrite both of a `visit_*` and `depart_*` pair.

+ If calling the parent `visit_*`, also call the parent `depart_*`,
e.g.::

def visit_example(self, node):
if foo:
do_something()
else:
_html_base.HTMLTranslator.visit_example(self, node)

def depart_example(self, node):
if foo:
do_your_own_cleanup()
else:
_html_base.HTMLTranslator.depart_example(self, node)

This way, changes in stack use will not bite you.
"""
and ::

class HTMLTranslator(writers._html_base.HTMLTranslator):
"""
This writer generates `polyglot markup`: HTML 5 that is also valid XML.

See the docstring of docutils.writers._html_base.HTMLTranslator for
safe subclassing.
"""

Also, how about defining/documenting `context` at class-level (still
ensuring it is an instance-specific list in __init__)?

@@ -136,6 +160,13 @@
}
"""Character references for characters with a special meaning in HTML."""

+ context = []
+ """Heterogeneous stack.
+
+ Used by visit_* and depart_* functions in conjunction with the tree
+ traversal. Make sure that the pops correspond to the pushes."""
+
+
def __init__(self, document):
nodes.NodeVisitor.__init__(self, document)
self.settings = settings = document.settings
@@ -168,10 +199,7 @@
self.math_output_options = self.math_output[1:]
self.math_output = self.math_output[0].lower()

- # A heterogenous stack used in conjunction with the tree traversal.
- # Make sure that the pops correspond to the pushes:
self.context = []
-
self.topic_classes = [] # TODO: replace with self_in_contents
self.colspecs = []
self.compact_p = True

If yes, should it be pre-instantiated to
a) a list (correct data-type) or
b) None (safeguard against shared use by several instances in case a
child class does not re-bind `self.context` in its __init__)?

...

Post by David Goodger
Hmm... docutils.writers.pep_html.HTMLTranslator defines
depart_field_list but not visit_field_list. Should we explicitly
define it as the following (for internal consistency
html4css1.HTMLTranslator.visit_field_list(self, node)

Generally, yes.
+1 set a good example
+1 safe us from surprises
-0 not necessary as we can detect errors with our test suite and
fix them easily.

Howver, in the above case the child's `depart_*` method unconditionally
calls the parent's `depart_*` method: this is safe already. If anything
at all, I would include the visit function as comment (the parent
function is called anyway)::

# ensure consistent use of the context stack
# def visit_field_list(self, node):
# html4css1.HTMLTranslator.visit_field_list(self, node)

Then, the recommended practice is already laid out for eventual later
changes.

Post by David Goodger
Other subclasses should be reviewed.

I did this for html5. In most cases, a copy of the partner-function
will be the most simple approach (see patch below).
"html4css1" and "xelatex" will follow.

Günter

Index: __init__.py
===================================================================
--- __init__.py (Revision 8012)
+++ __init__.py (Arbeitskopie)
@@ -148,11 +148,13 @@
class HTMLTranslator(writers._html_base.HTMLTranslator):
"""
This writer generates `polyglot markup`: HTML 5 that is also valid XML.
+
+ See the docstring of docutils.writers._html_base.HTMLTranslator for
+ safe subclassing.
"""
# def __init__(self, document):
# writers._html_base.HTMLTranslator.__init__(self, document)

-
# <acronym> tag not supported in HTML5. Use the <abbr> tag instead.
def visit_acronym(self, node):
# @@@ implementation incomplete ("title" attribute)
@@ -161,23 +163,34 @@
def depart_acronym(self, node):
self.body.append('</abbr>')

- # no meta tag in HTML 5
+ # no meta tag in HTML5
def visit_authors(self, node):
self.visit_docinfo_item(node, 'authors', meta=False)
+
+ def depart_authors(self, node):
+ self.depart_docinfo_item()

+ # no meta tag in HTML5
def visit_copyright(self, node):
self.visit_docinfo_item(node, 'copyright', meta=False)

- # no meta tag in HTML 5
+ def depart_copyright(self, node):
+ self.depart_docinfo_item()
+
+ # no meta tag in HTML5
def visit_date(self, node):
self.visit_docinfo_item(node, 'date', meta=False)

- # TODO: use HTML 5 <footer> element?
+ def depart_date(self, node):
+ self.depart_docinfo_item()
+
+ # TODO: use HTML5 <footer> element?
# def visit_footer(self, node):
# def depart_footer(self, node):

# TODO: use the new HTML5 element <aside>? (Also for footnote text)
# def visit_footnote(self, node):
+ # def depart_footnote(self, node):

# Meta tags: 'lang' attribute replaced by 'xml:lang' in XHTML 1.1
# HTML5/polyglot recommends using both
@@ -188,12 +201,20 @@
meta = self.emptytag(node, 'meta', **node.non_default_attributes())
self.add_meta(meta)

- # no meta tag in HTML 5
+ def depart_meta(self, node):
+ pass
+
+ # no meta tag in HTML5
def visit_organization(self, node):
self.visit_docinfo_item(node, 'organization', meta=False)

- # TODO: use the new HTML 5 element <section>?
+ def depart_organization(self, node):
+ self.depart_docinfo_item()
+
+ # TODO: use the new HTML5 element <section>?
# def visit_section(self, node):
+ # def depart_section(self, node):

# TODO: use the new HTML5 element <aside>?
# def visit_topic(self, node):
+ # def depart_topic(self, node):

David Goodger

2017-01-18 23:01:18 UTC

The latter option should definitely be part of the module documentation
(e.g. a "how to use this module" section in the module docstring).

How about amending the HTMLTranslator class docstring instead ...

...

"""Generic Docutils to HTML translator.
See the html4css1 and html5_polyglot for writers for full featured HTML
writers.
the `visit_*` and `depart_*` methods use a the
heterogeneous stack `self.context`.
+ Always overwrite both of a `visit_*` and `depart_*` pair.
+ If calling the parent `visit_*`, also call the parent `depart_*`,
do_something()
_html_base.HTMLTranslator.visit_example(self, node)
do_your_own_cleanup()
_html_base.HTMLTranslator.depart_example(self, node)

Sure. Some formatting quibbles (why a definition list? I'd rather see
"IMPORTANT" or similar).

Sometimes (often) the calls in depart should be in reverse order to
the calls in visit.

Post by Guenter Milde
This way, changes in stack use will not bite you.
"""
"""
This writer generates `polyglot markup`: HTML 5 that is also valid XML.
See the docstring of docutils.writers._html_base.HTMLTranslator for
safe subclassing.
"""
Also, how about defining/documenting `context` at class-level (still
ensuring it is an instance-specific list in __init__)?
@@ -136,6 +160,13 @@
}
"""Character references for characters with a special meaning in HTML."""
+ context = []

I *really* don't like this. A mutable attribute at class level is bad
practice, even if it's immediately replaced. Better would be to say

Post by Guenter Milde
+ """Heterogeneous stack.
+
+ Used by visit_* and depart_* functions in conjunction with the tree
+ traversal. Make sure that the pops correspond to the pushes."""

I'd rather put that docstring in the __init__ method. Specify it once,
in the right place.

Post by Guenter Milde
nodes.NodeVisitor.__init__(self, document)
self.settings = settings = document.settings
@@ -168,10 +199,7 @@
self.math_output_options = self.math_output[1:]
self.math_output = self.math_output[0].lower()
- # A heterogenous stack used in conjunction with the tree traversal.
self.context = []

Put the docstring here.

Post by Guenter Milde
self.topic_classes = [] # TODO: replace with self_in_contents
self.colspecs = []
self.compact_p = True
If yes, should it be pre-instantiated to
a) a list (correct data-type) or
b) None (safeguard against shared use by several instances in case a
child class does not re-bind `self.context` in its __init__)?

The correct choice (if yes) is (b), with documentation. But best is to
do it only once. So: no.

If a child class doesn't call the parent __init__, nothing will work.
I don't think we need to worry about that.

Generally, yes.
+1 set a good example
+1 safe us from surprises
-0 not necessary as we can detect errors with our test suite and
fix them easily.
Howver, in the above case the child's `depart_*` method unconditionally
calls the parent's `depart_*` method: this is safe already. If anything
at all, I would include the visit function as comment (the parent
# ensure consistent use of the context stack
# html4css1.HTMLTranslator.visit_field_list(self, node)
Then, the recommended practice is already laid out for eventual later
changes.

Yes, but this doesn't guarantee anything, or *do* anything. Bugs won't
be exposed.

You omitted the alternative::

def visit_field_list(self, node):
# ensure that we're not using the context stack:
context_stack_length = len(self.context)
html4css1.HTMLTranslator.visit_field_list(self, node)
# or 1 less, etc, depending on usage:
assert len(self.context) == context_stack_length

It may be a pain to implement, but at least this *does* something for
us. (Maybe this could be implemented as a decorator.)

Post by David Goodger
Other subclasses should be reviewed.

I did this for html5. In most cases, a copy of the partner-function
will be the most simple approach (see patch below).
"html4css1" and "xelatex" will follow.

-1. This violates the DRY principle (don't repeat yourself). If we
copy functions between parent classes and subclasses, what's the point
of having the parent classes?

David Goodger
<http://python.net/~goodger>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Guenter Milde

2017-01-19 11:03:32 UTC

Dear David,

thank you for the fast reply.

How about amending the HTMLTranslator class docstring instead ...

...

Post by David Goodger
Sure. Some formatting quibbles (why a definition list? I'd rather see
"IMPORTANT" or similar).

Post by David Goodger
Sometimes (often) the calls in depart should be in reverse order to
the calls in visit.

Yes, but this is just one example (the use case in Sphinx, whith custom
"visit code" for SVG images and the parent function for all other
images. See below for other examples.

...

Post by Guenter Milde
Also, how about defining/documenting `context` at class-level

...

Post by David Goodger
I *really* don't like this. A mutable attribute at class level is bad
practice, even if it's immediately replaced.

I fully agree.

Post by David Goodger
I'd rather put that docstring in the __init__ method. Specify it once,
in the right place.

Agreed.

Does it make sense to convert it from a comment to a docstring?
In other words: Is there an API-doc-generator that will find the
docstrings of function variables?

...

Generally, yes.
+1 set a good example
+1 safe us from surprises
-0 not necessary as we can detect errors with our test suite and
fix them easily.
Howver, in the above case the child's `depart_*` method unconditionally
calls the parent's `depart_*` method: this is safe already. If anything
at all, I would include the visit function as comment (the parent
# ensure consistent use of the context stack
# html4css1.HTMLTranslator.visit_field_list(self, node)
Then, the recommended practice is already laid out for eventual later
changes.

Yes, but this doesn't guarantee anything, or *do* anything. Bugs won't
be exposed.

It does not do anything, because no action is required in this case.

Post by David Goodger
context_stack_length = len(self.context)
html4css1.HTMLTranslator.visit_field_list(self, node)
assert len(self.context) == context_stack_length
It may be a pain to implement, but at least this *does* something for
us. (Maybe this could be implemented as a decorator.)

However, is it worth the pain and does it do the right thing?

a) There is already a stack-check at the end of the translation.
Bugs will be exposed. The additional check gives just a more precise
location of the problem.

b) In the discussed use case, the matching depart function is::

def depart_field_list(self, node):
html4css1.HTMLTranslator.depart_field_list(self, node)
if 'rfc2822' in node['classes']:
self.body.append('<hr />\n')

It calls the parent depart function unconditionally and does not use
the `context` stack.

If _html_base.HTMLTranslator.visit_field_list() and
_html_base.HTMLTranslator.depart_field_list() were changed to
use the `context` stack, this would not lead to errors or wrong output.

The additional check would just give false positives!

c) In a 3rd party application, the important point is not to spot changes
in context-stack use but to ensure the app keeps working with a new
Docutils version.

This can be implemented by ensuring that the parent `visit_*` function
is called (rsp. not called) by the onwn code under the same
conditions as the parent `depart_*` function.

Alternatives for robust coding:

a) Overwrite both, don't call the parent.

b) Overwrite both, call the parent::

def visit_field_list(self, node):
if foo:
self.body.append('<div class="foo">\n')
html4css1.HTMLTranslator.visit_field_list(self, node)

def depart_field_list(self, node):
html4css1.HTMLTranslator.depart_field_list(self, node)
if foo:
self.body.append('</div>\n')

c) Overwrite both, call the parent under the same conditions
(see the "Spinx image" example from the docstring).

d) Overwrite one, don't use the stack, call the parent
(the "PEP field_list" example above).

Post by David Goodger
Other subclasses should be reviewed.

I did this for html5. In most cases, a copy of the partner-function
will be the most simple approach (see patch below).
"html4css1" and "xelatex" will follow.

-1. This violates the DRY principle (don't repeat yourself).

Robust coding requires treating visit/depart pairs as a *unit*.

A common approach for configuring the behaviour of a class is to copy a
method's definition to the child class and modify it. In many cases, this
is simpler (and also less code) than calling the parent method.

Treating the visit/depart pair as a unit means then copying both methods
-- even if only one is modified. It is a judgement call between DRY and
KISS principles.

Post by David Goodger
If we copy functions between parent classes and subclasses, what's the
point of having the parent classes?

With 189 methods in the parent and 12 in the child, subclassing is still
justified ;-)

However, as we are speaking about Docutils and not a 3rd-party application,
I am indifferent regarding the review:

+1 internal consistency,
example for 3rd-party apps that shall work across Docutils versions.

-1 additional code, not required for the Docutils core.

Günter

David Goodger

2017-01-29 22:54:01 UTC

Post by Guenter Milde
Dear David,
thank you for the fast reply.

How about amending the HTMLTranslator class docstring instead ...

...

Post by David Goodger
Sure. Some formatting quibbles (why a definition list? I'd rather see
"IMPORTANT" or similar).

Post by David Goodger
Sometimes (often) the calls in depart should be in reverse order to
the calls in visit.

Yes, but this is just one example (the use case in Sphinx, whith custom
"visit code" for SVG images and the parent function for all other
images. See below for other examples.
...

Post by Guenter Milde
Also, how about defining/documenting `context` at class-level

...

Post by David Goodger
I *really* don't like this. A mutable attribute at class level is bad
practice, even if it's immediately replaced.

I fully agree.

Post by David Goodger
I'd rather put that docstring in the __init__ method. Specify it once,
in the right place.

Agreed.
Does it make sense to convert it from a comment to a docstring?

I think so. But I'm biased.

Post by Guenter Milde
In other words: Is there an API-doc-generator that will find the
docstrings of function variables?

I don't know if Sphinx or any other tool implements this. It is
described in PEP 258
(https://www.python.org/dev/peps/pep-0258/#attribute-docstrings). I
use this convention in my own code extensively, regardless of tooling.
See nodes.py for examples.

Generally, yes.
+1 set a good example
+1 safe us from surprises
-0 not necessary as we can detect errors with our test suite and
fix them easily.
Howver, in the above case the child's `depart_*` method unconditionally
calls the parent's `depart_*` method: this is safe already. If anything
at all, I would include the visit function as comment (the parent
# ensure consistent use of the context stack
# html4css1.HTMLTranslator.visit_field_list(self, node)
Then, the recommended practice is already laid out for eventual later
changes.

Yes, but this doesn't guarantee anything, or *do* anything. Bugs won't
be exposed.

It does not do anything, because no action is required in this case.

This precision is useful.

Post by Guenter Milde
html4css1.HTMLTranslator.depart_field_list(self, node)
self.body.append('<hr />\n')
It calls the parent depart function unconditionally and does not use
the `context` stack.
If _html_base.HTMLTranslator.visit_field_list() and
_html_base.HTMLTranslator.depart_field_list() were changed to
use the `context` stack, this would not lead to errors or wrong output.
The additional check would just give false positives!

I don't think so (but correct me if I'm wrong). If the wrapper (of the
depart method, say) has ``assert len(self.context) ==
context_stack_length`` and the implementation in the parent class (of
both visit & depart) changes, the assertion would fail. The visit
method overwrites (& removes) the changed behavior, but the depart
method doesn't, so the stack sizes won't match. The assertion is
testing the subclass' assumptions of the parent class implementation;
if those details change, the assertion fails.

Post by Guenter Milde
c) In a 3rd party application, the important point is not to spot changes
in context-stack use but to ensure the app keeps working with a new
Docutils version.
This can be implemented by ensuring that the parent `visit_*` function
is called (rsp. not called) by the onwn code under the same
conditions as the parent `depart_*` function.

"onwn code"?

Post by Guenter Milde
a) Overwrite both, don't call the parent.
self.body.append('<div class="foo">\n')
html4css1.HTMLTranslator.visit_field_list(self, node)
html4css1.HTMLTranslator.depart_field_list(self, node)
self.body.append('</div>\n')
c) Overwrite both, call the parent under the same conditions
(see the "Spinx image" example from the docstring).
d) Overwrite one, don't use the stack, call the parent
(the "PEP field_list" example above).

These options should be documented.

Post by David Goodger
Other subclasses should be reviewed.

I did this for html5. In most cases, a copy of the partner-function
will be the most simple approach (see patch below).
"html4css1" and "xelatex" will follow.

-1. This violates the DRY principle (don't repeat yourself).

Robust coding requires treating visit/depart pairs as a *unit*.
A common approach for configuring the behaviour of a class is to copy a
method's definition to the child class and modify it. In many cases, this
is simpler (and also less code) than calling the parent method.
Treating the visit/depart pair as a unit means then copying both methods
-- even if only one is modified. It is a judgement call between DRY and
KISS principles.

True. These options should be documented. The important thing is to
instill in developers of client code the need to treat visit/depart
pairs as a unit.

Post by David Goodger
If we copy functions between parent classes and subclasses, what's the
point of having the parent classes?

With 189 methods in the parent and 12 in the child, subclassing is still
justified ;-)
However, as we are speaking about Docutils and not a 3rd-party application,
+1 internal consistency,
example for 3rd-party apps that shall work across Docutils versions.
-1 additional code, not required for the Docutils core.

I think it would be useful to add this guard code to show by example.
Better yet, we could define a decorator that does this succinctly.

David Goodger
<http://python.net/~goodger>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Guenter Milde

2017-02-01 16:27:12 UTC

Dear David,
...

Post by David Goodger
I'd rather put that docstring in the __init__ method. Specify it once,
in the right place.

...

Post by Guenter Milde
Does it make sense to convert it from a comment to a docstring?

I think so. But I'm biased.

Post by Guenter Milde
In other words: Is there an API-doc-generator that will find the
docstrings of function variables?

I see an advantage in having a consistent documentation style.
Also, it may give a hint that this is more than Docutils-internal
documentation.

Generally, yes.
+1 set a good example
+1 safe us from surprises
-0 not necessary as we can detect errors with our test suite and
fix them easily.
Howver, in the above case the child's `depart_*` method unconditionally
calls the parent's `depart_*` method: this is safe already. If anything
at all, I would include the visit function as comment (the parent
# ensure consistent use of the context stack
# html4css1.HTMLTranslator.visit_field_list(self, node)
Then, the recommended practice is already laid out for eventual later
changes.

Yes, but this doesn't guarantee anything, or *do* anything. Bugs won't
be exposed.

It does not do anything, because no action is required in this case.

This precision is useful.

I don't think so (but correct me if I'm wrong).

Post by David Goodger
... docutils.writers.pep_html.HTMLTranslator defines
depart_field_list but not visit_field_list.

If the wrapper (of the depart method, say) has ``assert
len(self.context) == context_stack_length`` and the implementation in
the parent class (of both visit & depart) changes, the assertion would
fail. The visit method overwrites (& removes) the changed behavior, but
the depart method doesn't, so the stack sizes won't match. The
assertion is testing the subclass' assumptions of the parent class
implementation;

However, docutils.writers.pep_html.HTMLTranslator
*does not make any assumption* of the parent class implementation regarding
stack use:

* it calls the parents' visit_field_list() method (by not overwriting
it),
* it calls the parents' depart_field_list() method (in its local definition)
* it does not use the context stack.

Post by David Goodger
if those details change, the assertion fails.

... while the visit_field_list() / depart_field_list() pair continues to
work as expected.

This is what I call a false positive.

Asserting a detail *that is not relevant* is not just useless. It requires
to change the assertion when this irrelevant implementation detail
changes. (For Docutils core code, this is not too big a problem, for
3rd-party applications this establishes a needless dependency on a certain
Docutils version.)

"onwn code"?

own (local, child, ...) code

These options should be documented.

This would make a long docstring but I am fine with this.

Post by David Goodger
Other subclasses should be reviewed.

I did this for html5. In most cases, a copy of the partner-function
will be the most simple approach (see patch below).
"html4css1" and "xelatex" will follow.

-1. This violates the DRY principle (don't repeat yourself).

Robust coding requires treating visit/depart pairs as a *unit*.
A common approach for configuring the behaviour of a class is to copy a
method's definition to the child class and modify it. In many cases, this
is simpler (and also less code) than calling the parent method.
Treating the visit/depart pair as a unit means then copying both methods
-- even if only one is modified. It is a judgement call between DRY and
KISS principles.

True. These options should be documented. The important thing is to
instill in developers of client code the need to treat visit/depart
pairs as a unit.

Post by David Goodger
If we copy functions between parent classes and subclasses, what's the
point of having the parent classes?

I think it would be useful to add this guard code to show by example.

The guard code is already in my local version but you pointed out it
violates DRY.

So it is OK to commit?

Post by David Goodger
Better yet, we could define a decorator that does this succinctly.

Feel free to implement this. (I am no fan of decorators.)

Günter Milde

David Goodger

2017-02-13 05:20:52 UTC

Post by Guenter Milde
Dear David,
...

Post by David Goodger
I'd rather put that docstring in the __init__ method. Specify it once,
in the right place.

...

Post by Guenter Milde
Does it make sense to convert it from a comment to a docstring?

I think so. But I'm biased.

Post by Guenter Milde
In other words: Is there an API-doc-generator that will find the
docstrings of function variables?

I see an advantage in having a consistent documentation style.
Also, it may give a hint that this is more than Docutils-internal
documentation.

Generally, yes.
+1 set a good example
+1 safe us from surprises
-0 not necessary as we can detect errors with our test suite and
fix them easily.
Howver, in the above case the child's `depart_*` method unconditionally
calls the parent's `depart_*` method: this is safe already. If anything
at all, I would include the visit function as comment (the parent
# ensure consistent use of the context stack
# html4css1.HTMLTranslator.visit_field_list(self, node)
Then, the recommended practice is already laid out for eventual later
changes.

Yes, but this doesn't guarantee anything, or *do* anything. Bugs won't
be exposed.

It does not do anything, because no action is required in this case.

This precision is useful.

I don't think so (but correct me if I'm wrong).

Post by David Goodger
... docutils.writers.pep_html.HTMLTranslator defines
depart_field_list but not visit_field_list.

However, docutils.writers.pep_html.HTMLTranslator
*does not make any assumption* of the parent class implementation regarding
* it calls the parents' visit_field_list() method (by not overwriting
it),
* it calls the parents' depart_field_list() method (in its local definition)
* it does not use the context stack.

Then that's a bad example. Let's find a better one. Here's an example
similar to the one that started all this. Let's say that the PEP HTML
Writer overrides the visit_problematic method, but leaves
depart_problematic alone. In _html_base.HTMLTranslator, these methods
use the context stack. A diff patch is below. Attached is
context_check.py, which I propose adding to _html_base.py.

Index: pep_html/__init__.py
===================================================================
--- pep_html/__init__.py (revision 8028)
+++ pep_html/__init__.py (working copy)
@@ -16,6 +16,7 @@
import docutils
from docutils import frontend, nodes, utils, writers
from docutils.writers import html4css1
+from docutils.writers._html_base import context_check

class Writer(html4css1.Writer):
@@ -102,3 +103,15 @@
html4css1.HTMLTranslator.depart_field_list(self, node)
if 'rfc2822' in node['classes']:
self.body.append('<hr />\n')
+
+ def visit_problematic(self, node):
+ if node.hasattr('refid'):
+ self.body.append('<a href="#%s">' % node['refid'])
+ self.context.append('</a>')
+ else:
+ self.context.append('')
+ self.body.append(self.starttag(node, 'span', '', CLASS='problematic'))
+
+ @context_check(-1)
+ def depart_problematic(self, node):
+ html4css1.HTMLTranslator.depart_problematic(self, node)

In the diff, you'll see that there's a depart_problematic, guarded
with a context_check(-1) decorator. The -1 means that the context
stack size is expected to decrease by 1 after this code is run. The
visit_problematic method doesn't call the superclass's method; it's a
pure override. The context stack usage is balanced though; one item is
pushed at the visit, and one item is popped at the depart.

Now let's say that in the superclass (_html_base.HTMLTranslator), the
visit/depart_problematic methods have been modified such that they no
longer use the context stack. The PEP pair of methods are no longer
balanced wrt their use of the context stack. The guard code in the
sample above will definitely generate an exception, pinpointing the
issue.

Had code such as the above been in place, the issue with Sphinx would
have been easily and immediately identified.

Post by David Goodger
if those details change, the assertion fails.

... while the visit_field_list() / depart_field_list() pair continues to
work as expected.
This is what I call a false positive.
Asserting a detail *that is not relevant* is not just useless. It requires
to change the assertion when this irrelevant implementation detail
changes. (For Docutils core code, this is not too big a problem, for
3rd-party applications this establishes a needless dependency on a certain
Docutils version.)

I think we've been misunderstanding each other. Look at the example
code I provide above, and critique that please.

That's exactly what happened with Sphinx though.

Post by David Goodger
I think it would be useful to add this guard code to show by example.

The guard code is already in my local version but you pointed out it
violates DRY.
So it is OK to commit?

Please show it first so I know exactly what you're proposing.

Post by David Goodger
Better yet, we could define a decorator that does this succinctly.

Feel free to implement this. (I am no fan of decorators.)

Attached (context_check.py). It requires Python 2.5.

Decorators can be very useful.

David Goodger
<http://python.net/~goodger>

Guenter Milde

2017-02-14 13:19:10 UTC

...

Here's an example similar to the one that started all this. Let's say
that the PEP HTML Writer overrides the visit_problematic method, but
leaves depart_problematic alone.
In _html_base.HTMLTranslator, these methods
use the context stack. A diff patch is below. Attached is
context_check.py, which I propose adding to _html_base.py.
Index: pep_html/__init__.py
===================================================================
--- pep_html/__init__.py (revision 8028)
+++ pep_html/__init__.py (working copy)
@@ -16,6 +16,7 @@
import docutils
from docutils import frontend, nodes, utils, writers
from docutils.writers import html4css1
+from docutils.writers._html_base import context_check
@@ -102,3 +103,15 @@
html4css1.HTMLTranslator.depart_field_list(self, node)
self.body.append('<hr />\n')
+
+ self.body.append('<a href="#%s">' % node['refid'])
+ self.context.append('</a>')
+ self.context.append('')
+ self.body.append(self.starttag(node, 'span', '', CLASS='problematic'))
+
+ html4css1.HTMLTranslator.depart_problematic(self, node)
In the diff, you'll see that there's a depart_problematic, guarded
with a context_check(-1) decorator. The -1 means that the context
stack size is expected to decrease by 1 after this code is run. The
visit_problematic method doesn't call the superclass's method; it's a
pure override. The context stack usage is balanced though; one item is
pushed at the visit, and one item is popped at the depart.

Defensive programming would define the depart_problematic method in PEP as

+ def depart_problematic(self, node):
+ self.body.append('</span>')
+ self.body.append(self.context.pop())

Now let's say that in the superclass (_html_base.HTMLTranslator), the
visit/depart_problematic methods have been modified such that they no
longer use the context stack. The PEP pair of methods are no longer
balanced wrt their use of the context stack. The guard code in the
sample above will definitely generate an exception, pinpointing the
issue.

... while the safe subclassing (either call both upstream visit_* and
depart_* or overwrite both) would ensure balanced stack use independent
of the superclass change.

Look at the example code I provide above, and critique that please.

The example is slightly better in that is does not trigger a spurious
exception.

But I still do not see a convincing use case:

What is the use of "an exception, pinpointing the issue" whe we can easily
avoid the issue?

The "guard code" does not help the users of 3rd-party applications. The
Docutils change will still break the application! OTOH, with safe
subclassing, the change in Docutils will not break the 3rd-party
application.

Had code such as the above been in place, the issue with Sphinx would
have been easily and immediately identified.

...

There are several points here:

1. We did not think about context stack use (now it is part of the
documentation and mindset).

2. We did not give a pre-release to Sphinx for testing (now added to
our release procedure steps).

So, yes,

* if the Sphinx developers had put in place such code and
* if at least one Sphinx user had tested Sphinx with a Docutils
pre-release (or the devel version)

the issue with Sphinx could have been easier identified.

However, with the right problem awareness, Sphinx developers could have
put in place the safe subclassing of visit/depart method pairs and the
problem would not even have occured!

With

c) Overwrite both, call the parent functions under the same
conditions::

def visit_example(self, node):
if foo:
<my special code>
else: # call the parent method
_html_base.HTMLTranslator.visit_example(self, node)

def depart_example(self, node):
if foo:
<my special code>
else: # call the parent method
_html_base.HTMLTranslator.depart_example(self, node)

Sphinx can be used with Docutils 0.12 and 0.13. No need for code that
generates an exception.

Summary:
I don't see a convincing use case for code to "easily and immediately
identify a problem" when the same problem can just as easily be
prevented.

=========================================================================

...

Post by David Goodger
I think it would be useful to add this guard code to show by example.

The guard code is already in my local version but you pointed out it
violates DRY.
So it is OK to commit?

Please show it first so I know exactly what you're proposing.

It seems there was a misunderstanding regarding what can be regarded
"guard code example":

I propose to give a good example by using "safe subclassing" as safeguard
against context stack problems --- even if this implies a bit of code
duplication (see below).

I agree that we should avoid code duplication, but not at any cost.

Günter

Exec: svn 'diff' '__init__.py' 2>&1
Dir: docutils/docutils/writers/html5_polyglot/

Index: __init__.py
===================================================================
--- __init__.py (Revision 8036)
+++ __init__.py (Arbeitskopie)
@@ -148,11 +148,13 @@
class HTMLTranslator(writers._html_base.HTMLTranslator):
"""
This writer generates `polyglot markup`: HTML5 that is also valid XML.
+
+ See the docstring of docutils.writers._html_base.HTMLTranslator for
+ safe subclassing.
"""
# def __init__(self, document):
# writers._html_base.HTMLTranslator.__init__(self, document)

-
# <acronym> tag not supported in HTML5. Use the <abbr> tag instead.
def visit_acronym(self, node):
# @@@ implementation incomplete ("title" attribute)
@@ -165,13 +167,23 @@
def visit_authors(self, node):
self.visit_docinfo_item(node, 'authors', meta=False)

+ def depart_authors(self, node):
+ self.depart_docinfo_item()
+
+ # no meta tag in HTML5
def visit_copyright(self, node):
self.visit_docinfo_item(node, 'copyright', meta=False)

+ def depart_copyright(self, node):
+ self.depart_docinfo_item()
+
# no meta tag in HTML5
def visit_date(self, node):
self.visit_docinfo_item(node, 'date', meta=False)

+ def depart_date(self, node):
+ self.depart_docinfo_item()
+
# TODO: use HTML5 <footer> element?
# def visit_footer(self, node):
# def depart_footer(self, node):
@@ -178,6 +190,7 @@

# TODO: use the new HTML5 element <aside>? (Also for footnote text)
# def visit_footnote(self, node):
+ # def depart_footnote(self, node):

# Meta tags: 'lang' attribute replaced by 'xml:lang' in XHTML 1.1
# HTML5/polyglot recommends using both
@@ -188,12 +201,20 @@
meta = self.emptytag(node, 'meta', **node.non_default_attributes())
self.add_meta(meta)

+ def depart_meta(self, node):
+ pass
+
# no meta tag in HTML5
def visit_organization(self, node):
self.visit_docinfo_item(node, 'organization', meta=False)

+ def depart_organization(self, node):
+ self.depart_docinfo_item()
+
# TODO: use the new HTML5 element <section>?
# def visit_section(self, node):
+ # def depart_section(self, node):

# TODO: use the new HTML5 element <aside>?
# def visit_topic(self, node):
+ # def depart_topic(self, node):

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply A

David Goodger

2017-02-14 23:09:08 UTC

Defensive programming would define the depart_problematic method in PEP as
+ self.body.append('</span>')
+ self.body.append(self.context.pop())

What if depart_problematic is a lot of code, and the client code is
making no changes? They should not duplicate the code. Even two or
three lines of code shouldn't be duplicated if there's no underlying
change.

... while the safe subclassing (either call both upstream visit_* and
depart_* or overwrite both) would ensure balanced stack use independent
of the superclass change.

Again, what if the client is only changing one of these? This will
happen. This has happened.

Look at the example code I provide above, and critique that please.

The example is slightly better in that is does not trigger a spurious
exception.

It will trigger an exception if the underlying conditions change.

Post by Guenter Milde
What is the use of "an exception, pinpointing the issue" whe we can easily
avoid the issue?

Because we won't always be able to avoid the issue. You seem to be of
the opinion that we can force client code to always either override
both visit/depart or extend both. This is **not true**. Some client
code will want to override one and either extend the other or just use
the other unchanged. I am simply presenting **another option** that
will cover these cases.

Post by Guenter Milde
The "guard code" does not help the users of 3rd-party applications. The
Docutils change will still break the application!

Which is better: (1) a 3rd-party application breaks with a clear
indication of where and why, or (2) it breaks with no such indication.
#1 is obviously better. That's what this provides.

Post by Guenter Milde
OTOH, with safe
subclassing, the change in Docutils will not break the 3rd-party
application.

We **cannot** enforce such "safe subclassing". We can and should
encourage it, but some client code will be unable to use it.

Had code such as the above been in place, the issue with Sphinx would
have been easily and immediately identified.

...
1. We did not think about context stack use (now it is part of the
documentation and mindset).
2. We did not give a pre-release to Sphinx for testing (now added to
our release procedure steps).
So, yes,
* if the Sphinx developers had put in place such code and
* if at least one Sphinx user had tested Sphinx with a Docutils
pre-release (or the devel version)
the issue with Sphinx could have been easier identified.
However, with the right problem awareness, Sphinx developers could have
put in place the safe subclassing of visit/depart method pairs and the
problem would not even have occured!

If you think that all client code will implement "safe subclassing",
you're deluding yourself.

I am not talking about replacing the documentation about proper use.
I'm just talking about adding another option.

Post by Guenter Milde
With
c) Overwrite both, call the parent functions under the same

BTW, it's "override", not "overwrite". Two different things.

Post by Guenter Milde
<my special code>
else: # call the parent method
_html_base.HTMLTranslator.visit_example(self, node)
<my special code>
else: # call the parent method
_html_base.HTMLTranslator.depart_example(self, node)
Sphinx can be used with Docutils 0.12 and 0.13. No need for code that
generates an exception.

Again, I think it's delusional to think that this is always going to happen.

Post by Guenter Milde
I don't see a convincing use case for code to "easily and immediately
identify a problem" when the same problem can just as easily be
prevented.

If it can be prevented, great. I don't think it always can. I'm just
proposing another option, another tool for client code developers to
use.

My proposed patch is attached.

I don't understand the pushback on this. At first it was, "that cannot
work". I showed that it can work. Now it's "I don't see a convincing
case". I've tried to convince you. If you are not yet convinced, then
I don't know what else I can do. So let's turn it around: I believe
this is a beneficial addition, or at worst harmless. What harm do you
think this could introduce?

Post by Guenter Milde
=========================================================================
...

Post by David Goodger
I think it would be useful to add this guard code to show by example.

The guard code is already in my local version but you pointed out it
violates DRY.
So it is OK to commit?

Please show it first so I know exactly what you're proposing.

It seems there was a misunderstanding regarding what can be regarded
I propose to give a good example by using "safe subclassing" as safeguard
against context stack problems --- even if this implies a bit of code
duplication (see below).

+1 on your patch. The Docutils code should exemplify good practice by
following our own guidelines.

Please add an explanatory comment before each piece of duplicate code:
# intentional duplicate of superclass code to maintain visit/depart

Post by Guenter Milde
I agree that we should avoid code duplication, but not at any cost.

What cost?
How many lines of code before the code duplication itself becomes too expensive?
How is one more expensive than the other?

David Goodger
<http://python.net/~goodger>

Guenter Milde

2017-02-17 20:57:44 UTC

Dear David,
...

Post by David Goodger
We **cannot** enforce such "safe subclassing". We can and should
encourage it, but some client code will be unable to use it.

Let' remember the problem at the start of this thread:
The discussion was "What is part of the API?" and the exmple

Post by Guenter Milde
Docutils changes the visit/depart method pair not to use the context stack.

...

Post by David Goodger
But what about when we change a visit/depart method pair to use
the context stack, when formerly they didn't? There's no way to keep
"side effect API" backwards compatibility in this case. So I think we
should not guarantee such backwards compatibility.

If we do not guarantee such compatibility, then internal changes may break
client code, if it depends on stack use details.

...

Post by David Goodger
I don't understand the pushback on this. At first it was, "that cannot
work". I showed that it can work.

It depends on the definition of "work". Your code does not prevent
internal changes breaking client code.

Post by David Goodger
Now it's "I don't see a convincing case". I've tried to convince you.
If you are not yet convinced, then I don't know what else I can do.

Provide a real use case.

The first example was harmfull, it introduced a dependency on
internal details where bfore there was none.

The second example was constructed, it showed where a dependency on
internal details *might* occur -- it copied a visit method (without any
changes!) and called the original depart. However, the original depart
was just 3 short lines, so copying it as well would be the better choice:
avoiding the dependency on internal details at low cost.

Now I am told: "But there are functions where copying both is too costly."

I don't challenge this. However, without a real use case, I cannot tell
whether there are other and better solutions (e.g. avoiding spaghetti
code in Docutils' visit/depart functions).

The addition could be useful for Docutils itself and tightly coupled
3d-party code - whenever it is no problem to keep versions in sync.
However, it still requires client code to add the decorator (see below
for discussion of an alternative).

Post by David Goodger
So let's turn it around: I believe this is a beneficial addition, or at
worst harmless. What harm do you think this could introduce?

As aspiring Docutils developer, I was taught: "Don't add anything without
a convincing use case." This is a good method to keep the code simple and
manageable. Put around: even a "harmless" addition bloats the code.

Post by David Goodger
I think it would be useful to add this guard code to show by example.

It is harmfull if we add this as a usage example without proper caveats:
Then, even following recommended use, client applications may break due
to internal changes in Docutils!

...

Post by Guenter Milde
I agree that we should avoid code duplication, but not at any cost.

What cost?

User frustration when a Docutils update breaks an application.

Post by David Goodger
How many lines of code before the code duplication itself becomes too expensive?
How is one more expensive than the other?

This is up to the client code developer to decide. I cannot tell once and
for all how many code lines it is worth to prevent the risk of breakage
with a Docutils update.

...

Attached is context_check.py, which I propose adding to
_html_base.py.

Actually, a context stack is not only used in _html_base, but also for
LaTeX, and Man writers.
If adding a context_check() method, why not do this (and the context
stack setup) in nodes.py?

We could even consider a stack-level check with every visit/depart call
there. Then, every unbalanced stack use could be localized without the
need to change client code.

Günter

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to re

Guenter Milde

2017-02-28 11:07:41 UTC

Dear David,
...

Attached is context_check.py, which I propose adding to
_html_base.py.

I tried this, but it resulted in numerous false positives of the kind

(ERROR/3) Stack size inconsistent: 1 before visiting node "title", 0 after departure.
(ERROR/3) Stack size inconsistent: 1 before visiting node "footer", 0 after departure.
(ERROR/3) Stack size inconsistent: 1 before visiting node "decoration", 0 after departure.
(ERROR/3) Stack size inconsistent: 1 before visiting node "title", 0 after departure.

I am not sure where my error is:

a) in the assumption that after a depart* call the stack should always be
at the same level as before the corresponding visit* call,

b) somewhere in the implementation.

Günter

Exec: svn 'diff' 'writers/_html_base.py' 'nodes.py' 2>&1
Dir: docutils/docutils

Index: writers/_html_base.py
===================================================================
--- writers/_html_base.py (Revision 8037)
+++ writers/_html_base.py (Arbeitskopie)
@@ -93,7 +93,7 @@
self.parts[part] = ''.join(getattr(self, part))

-class HTMLTranslator(nodes.NodeVisitor):
+class HTMLTranslator(nodes.ContextNodeVisitor):
"""Generic Docutils to HTML translator.

See the `html4css1` and `html5_polyglot` writers for full featured
@@ -182,7 +182,7 @@

def __init__(self, document):
- nodes.NodeVisitor.__init__(self, document)
+ nodes.ContextNodeVisitor.__init__(self, document)
self.settings = settings = document.settings
lcode = settings.language_code
self.language = languages.get_language(lcode, document.reporter)
@@ -213,7 +213,7 @@
self.math_output_options = self.math_output[1:]
self.math_output = self.math_output[0].lower()

- self.context = []
+ # self.context = []
"""Heterogeneous stack.

Used by visit_* and depart_* functions in conjunction with the tree
Index: nodes.py
===================================================================
--- nodes.py (Revision 8037)
+++ nodes.py (Arbeitskopie)
@@ -1929,6 +1929,44 @@
"""

+class ContextNodeVisitor(NodeVisitor):
+
+ """
+ Base class for traversals, where visit methods may store some information
+ in a heterogeneous "context" stack.
+
+ See `_html_base.HTMLTranslator` for a usage example.
+ """
+
+
+ def __init__(self, document):
+ NodeVisitor.__init__(self, document)
+ self.context = []
+ """Heterogeneous stack.
+
+ Used by visit_* and depart_* functions in conjunction with the tree
+ traversal. Make sure that the pops correspond to the pushes."""
+ # Store the stack-size befor/after visit/depart calls.
+ # Used to check balanced use of the "context" stack.
+ self._stack_size_before = 0
+
+
+ def dispatch_visit(self, node):
+ """Variant of dispatch_visit with context lenght check."""
+ self._stack_size_before = len(self.context)
+ NodeVisitor.dispatch_visit(self, node)
+
+ def dispatch_departure(self, node):
+ """Variant of dispatch_departure with context lenght check."""
+
+ result = NodeVisitor.dispatch_departure(self, node)
+ if self._stack_size_before != len(self.context):
+ self.document.reporter.error('Stack size inconsistent: '
+ '%i before visiting node "%s", %i after departure.'
+ % (self._stack_size_before, node.__class__.__name__,
+ len(self.context)))
+ return result
+
class GenericNodeVisitor(NodeVisitor):

"""

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to r

Guenter Milde

2017-02-02 16:10:23 UTC

...

Post by Guenter Milde
Does it make sense to convert it from a comment to a docstring?

I think so. But I'm biased.

...

These options should be documented.

Done (in the HTMLTranslator class docstring, so they can be seen with e.g.
Pythons built-in help).

See Revision 8020.

Günter Milde

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" t

Guenter Milde

2017-01-18 07:25:37 UTC

Post by Guenter Milde
Also, clear rules on how to implement backwards incompatible changes
(announce time, pre-release ...) would be good.

Adding and announcing a release candidate (eventually maybe alpha &
beta releases too) before the final release would be useful too. Maybe
that's what you meant by pre-release.

Yes:

Some projects use an "alpha, beta, release candidate" pre-release cycle
to support testing by their users prior to a final release.

-- https://www.python.org/dev/peps/pep-0440/#pre-releases

And our developer documentation since the last problems has

Announce the upcoming release at the Sphinx-devel mailing list and ask
for testing with Sphinx.

-- http://docutils.sourceforge.net/docs/dev/release.html

Maybe this should be supplemented by a note in the Docutils project
policies, too.

Günter

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to t

David Goodger

2017-01-18 16:24:57 UTC

Post by Guenter Milde
Also, clear rules on how to implement backwards incompatible changes
(announce time, pre-release ...) would be good.

Adding and announcing a release candidate (eventually maybe alpha &
beta releases too) before the final release would be useful too. Maybe
that's what you meant by pre-release.

Some projects use an "alpha, beta, release candidate" pre-release cycle
to support testing by their users prior to a final release.
-- https://www.python.org/dev/peps/pep-0440/#pre-releases
And our developer documentation since the last problems has
Announce the upcoming release at the Sphinx-devel mailing list and ask
for testing with Sphinx.
-- http://docutils.sourceforge.net/docs/dev/release.html
Maybe this should be supplemented by a note in the Docutils project
policies, too.

It's a release-specific issue. The correct place is in the release notes.

David Goodger
<http://python.net/~goodger>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Guenter Milde

2017-01-18 07:00:15 UTC

Dear David,

We should consider some clean-up first, e.g.
* remove the `handle_io_errors` option from io.FileInput/Output.

handle_io_errors() is redundant and ignored
since Release 0.10 (2012-12-16) but

.. used by Sphinx up to version 1.3.1,
see https://github.com/sphinx-doc/sphinx/issues/1834

Fixed in Sphinx Release 1.3.2 (released Nov 29, 2015).
Sphinx is now at 1.5.1 (stable) and 1.6 (dev).

More than a year on and with the announcement still in place, is
it OK to finally remove the redundant code?

Günter

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All"

David Goodger

2017-01-18 16:06:03 UTC

Post by Guenter Milde
Dear David,

We should consider some clean-up first, e.g.
* remove the `handle_io_errors` option from io.FileInput/Output.

handle_io_errors() is redundant and ignored
since Release 0.10 (2012-12-16) but

.. used by Sphinx up to version 1.3.1,
see https://github.com/sphinx-doc/sphinx/issues/1834

Fine with me. Although a true deprecation should have included a
deprecation warning in the code itself. What guarantee do we have that
no other client code uses this?

This should be accompanied by a release note that (1) specifies the
removal and (2) states that Docutils version X (the released version)
is not compatible with Sphinx versions up to 1.3.1.

David Goodger
<http://python.net/~goodger>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Guenter Milde

2017-01-18 20:47:42 UTC

Post by Guenter Milde
Dear David,

...

Post by Guenter Milde
handle_io_errors() is redundant and ignored
since Release 0.10 (2012-12-16) but

.. used by Sphinx up to version 1.3.1,
see https://github.com/sphinx-doc/sphinx/issues/1834

Fine with me. Although a true deprecation should have included a
deprecation warning in the code itself. What guarantee do we have that
no other client code uses this?

We have no guarantee but as the previous time only Sphinx complained, there
is a high probability that the others either did not use the option or get
the message.

Post by David Goodger
This should be accompanied by a release note that (1) specifies the
removal and (2) states that Docutils version X (the released version)
is not compatible with Sphinx versions up to 1.3.1.

I would plan this for 1.0 then, not the bugfix 0.13.2.

Günter

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Plea

David Goodger

2017-01-18 23:03:17 UTC

Post by Guenter Milde
Dear David,

...

Post by Guenter Milde
handle_io_errors() is redundant and ignored
since Release 0.10 (2012-12-16) but

.. used by Sphinx up to version 1.3.1,
see https://github.com/sphinx-doc/sphinx/issues/1834

Fine with me. Although a true deprecation should have included a
deprecation warning in the code itself. What guarantee do we have that
no other client code uses this?

We have no guarantee but as the previous time only Sphinx complained, there
is a high probability that the others either did not use the option or get
the message.

I would plan this for 1.0 then, not the bugfix 0.13.2.

+1

David Goodger
<http://python.net/~goodger>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Guenter Milde

2017-01-18 07:21:05 UTC

Dear Brecht and co-devels,

On 2016-12-23, Brecht Machiels wrote:

...

Post by Brecht Machiels
if you're serious about using semver, docutils should already be 1.0.0
since the API (however you define it) has been stable and widely used
for many years now.

`Docutils release policy`__ implements the rules of semantic versioning

Since the request https://sourceforge.net/p/docutils/feature-requests/50/
the even more detailled `pip version rules` `PEP 440`__ are
part of our `release procedure`__

So, now we are at Docutils 0.13.2a [repository]

__ http://docutils.sourceforge.net/docs/dev/policies.html#version-numbering
__ https://www.python.org/dev/peps/pep-0440/
__ http://docutils.sourceforge.net/docs/dev/release.html

Should we add note on PEP 444 compliance to the Docutils release policy to
make this "official"?

Günter

David Goodger

2017-01-18 16:24:04 UTC

Post by Guenter Milde
Dear Brecht and co-devels,
...

Post by Brecht Machiels
if you're serious about using semver, docutils should already be 1.0.0
since the API (however you define it) has been stable and widely used
for many years now.

`Docutils release policy`__ implements the rules of semantic versioning
Since the request https://sourceforge.net/p/docutils/feature-requests/50/
the even more detailled `pip version rules` `PEP 440`__ are
part of our `release procedure`__
So, now we are at Docutils 0.13.2a [repository]
__ http://docutils.sourceforge.net/docs/dev/policies.html#version-numbering
__ https://www.python.org/dev/peps/pep-0440/
__ http://docutils.sourceforge.net/docs/dev/release.html
Should we add note on PEP 444 compliance to the Docutils release policy to
make this "official"?

Sure, if you think that would be helpful. Requiring somebody to read
the entirety of PEP 444 is not helpful.

Beware the trap of over-specifying and too much formalism: these notes
and specifications should be helpful and enable quick development, not
be a burden.

David Goodger
<http://python.net/~goodger>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

David Goodger

2017-01-02 21:17:58 UTC

All of the above. Unless something is explicitly marked "private" or
"internal only", it's part of the public API.

In this case, I suggest a review looking for auxiliary functions that we may
want to exclude from the API before release 1.x.

Should the next release be 1.0.0 then?
Probably.
(½ ;-)

Was there ever any discussion of these items? I can only find their
SVN commit messages.

The issues came up such a long time ago, I don't know how explicitly they
were discussed in the list. In any case, they are in todo.txt since
oct. 2009.

Right: added by you, without any discussion that I can find. Just
because some text has been there for so long doesn't give it any
gravity. I added my notes to the "Doctree pruning" section in
todo.txt.

Post by David Goodger
What is the justification for these removals?

Post by David Goodger
Without justification: If it ain't broke, don't fix it.

+1 writers would not need special visit/depart methods for all admonition
types.
+1 doctest-blocks would become a special case of code and "naturally" gain
from the syntax highlight.

-1: <note>{body}</> is much more concise and expressive than
<admonition class="docutils-note"><title>Note</>{body}</>, and the
title translation can be put off until much later in the process.

Perhaps we should release a 0.13.2 or 0.14 which fixes the
incompatibility? (Followed by 1.0.0?) Of course, we'd need a volunteer
to actually implement the fix.

Try the patch at
https://sourceforge.net/p/docutils/bugs/_discuss/thread/48306ff1/d700/attachment/colwidth-auto-safe.patch

Now that the 0.13.1 version of those methods is published, wouldn't
reverting to the previous version potentially break code that has
adapted? If I were writing client code and this happened, I'd be
really frustrated and angry.
I suggest that first we get the opinion of Dmitry Shachnev who
reported the issue, so that this "fix" doesn't come back to bite him.
If he's OK with it, we should restore the old API with a bugfix
release, i.e. 0.13.2.

OK.
I checked Dmitries patches, they are safe with both.)
Reverting the "context" stack use in html4css1 for Sphinx is not safe with
the original patch there.

We haven't heard anything more from Dmitry, so we can only assume the
patch is fine with him. Go ahead and apply it.

Also there's the patch in the Jean Baptiste Favre "Migration docutils
0.12 to 0.13". Assuming that fixes the problem, it should also go into
0.13.2 ASAP.

David Goodger
<http://python.net/~goodger>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply

Dmitry Shachnev

2017-01-03 10:54:19 UTC

Hi David,

Post by Guenter Milde
I checked Dmitries patches, they are safe with both.)
Reverting the "context" stack use in html4css1 for Sphinx is not
safe with the original patch there.

We haven't heard anything more from Dmitry, so we can only assume the
patch is fine with him. Go ahead and apply it.

Yes, reverting the change to get_column_widths() is fine for me.
(Not reverting is also fine.)

--
Dmitry Shachnev

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

Jeffrey C. Jacobs

2016-12-23 23:27:28 UTC

Date: Fri, 23 Dec 2016 23:06:52 +0100
Subject: Re: [Docutils-develop] 1.0.0 & API
Content-Type: text/plain; charset=iso-8859-1; format=flowed
It would be good to more clearly define what defines the API exactly. I
would assume that all public functions/methods/attributes are part of
the API, for example. But does the docutils codebase differentiate
between private (underscore) and public symbols? Once 1.0.0 is
released, this definition will determine which changes go in 1.1.0 and
which go in 2.0.0.

I think you've hit the nail on the head here Brecht, what is the API.
Instead of considering whether we've reached 1.0.0, maybe the question
we should be asking is, is our documentation formal and complete
enough that if someone wanted to implement Docutils in C++ or Java
(not Jython!) they could take that spec and run with it, and it would
work for all cases just like the current python implementation. As
I'm working on a new document spec, I will say, such things are
tedious in the short run but pay off in the long run if formatted
properly, even if just as an extended PEP. And if we go all the way
to formal, ISO certification, which I know we have no intention of
doing at this time, then a secondary, third-party, reference
implementation would be necessary to prove it's a legitimate standard.

My point isn't though to write ISO-level standards, just to have
something in place we can point to to say, this is what Docutils 1.0.0
should look like.

Jeffrey.

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please use "Reply All" to reply to the list.

engelbert gruber

2016-12-24 10:19:10 UTC

ad 1.0.0 : the only argument is our own standard of stability (thanks
david.g)

overwriting open without and leaving close is ... not our fault
it is a kids learning process)

ad specification/documentation: i really dig especially in the form of
unittests

executable self checking specifications. the interesting part is having
them
crosslanguage usable)

ad ... very nice days to all

Guenter Milde

2016-12-30 22:01:33 UTC

Post by Jeffrey C. Jacobs

We have to differentiate between Docutils and rST here:

rST is defined formally and independent of the Docutils implementation.
There are already alternative implementations, e.g. "prest" in the
Docutils sandbox.

Post by Jeffrey C. Jacobs
As I'm working on a new document spec, I will say, such things are
tedious in the short run but pay off in the long run if formatted
properly, even if just as an extended PEP.

Unfortunately, the current document model description is broken, e.g. ::

#> xmllint --dtdvalid docutils.dtd transforms.xml

shows problems like ::

validity warning : Attribute align of element entry: already defined
%tbl.entry.att;
^
morecols NMTOKEN #IMPLIED
^

Post by Jeffrey C. Jacobs
My point isn't though to write ISO-level standards, just to have
something in place we can point to to say, this is what Docutils 1.0.0
should look like.

Fixing the dtd, we would have something to point what reStructuredText 1.0
should look like.

For Docutils, there is the additional requirement to allow stable use of the
code

a) via command line interface
b) as library

Günter

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Docutils-develop mailing list
Docutils-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/docutils-develop

Please u

David Goodger

2017-01-02 22:14:01 UTC

Post by Jeffrey C. Jacobs

Why should we care? I don't.

Post by Guenter Milde
rST is defined formally and independent of the Docutils implementation.

Not really. They're quite interdependent. There has never been
explicit support for other implementations, nor has the reST spec been
formally defined. It's quite informal now. reST is defined by the
combination of the spec documentation itself, the test cases, and the
code.

Post by Guenter Milde
There are already alternative implementations, e.g. "prest" in the
Docutils sandbox.

It's an implementation of a de-facto standard, not a formal standard.