[Docutils-develop] [docutils:bugs] #317 Extra space inserted with French smartquotes

[Dmitry Shachnev]

---

** [bugs:#317] Extra space inserted with French smartquotes**

**Status:** open
**Group:** Default
**Created:** Sun Apr 23, 2017 01:17 PM UTC by Dmitry Shachnev
**Last Updated:** Sun Apr 23, 2017 01:17 PM UTC
**Owner:** nobody


Another issue [spotted by @jfbu](https://github.com/sphinx-doc/sphinx/pull/3562#issuecomment-296399918). This command:

echo '(``"entry"``)' | ./tools/rst2xml.py --smart-quotes=yes --language=fr

produces:

<paragraph>(<literal>"entry"</literal> )</paragraph>

Note the extra NBSP (U+00A0) after the `</literal>`.

Dropping either `--language=fr` or `--smart-quotes=yes` makes it disappear.

With the release version (0.13.1), U+202F is inserted instead of U+00A0.


---

Sent from sourceforge.net because docutils-***@lists.sourceforge.net is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Günter Milde]

- **status**: open --> closed-fixed
- **Comment**:

Thanks for the report and analysis.

Fixed in Revision 8061.
Also for some other instances (quotes immediately before inline markup: "*foo*".



---

** [bugs:#317] Extra space inserted with French smartquotes**

**Status:** closed-fixed
**Group:** Default
**Created:** Sun Apr 23, 2017 01:17 PM UTC by Dmitry Shachnev
**Last Updated:** Sun Apr 23, 2017 01:17 PM UTC
**Owner:** nobody


Another issue [spotted by @jfbu](https://github.com/sphinx-doc/sphinx/pull/3562#issuecomment-296399918). This command:

echo '(``"entry"``)' | ./tools/rst2xml.py --smart-quotes=yes --language=fr

produces:

<paragraph>(<literal>"entry"</literal> )</paragraph>

Note the extra NBSP (U+00A0) after the `</literal>`.

Dropping either `--language=fr` or `--smart-quotes=yes` makes it disappear.

With the release version (0.13.1), U+202F is inserted instead of U+00A0.


---

Sent from sourceforge.net because docutils-***@lists.sourceforge.net is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Dmitry Shachnev]

Thank you!

While trying to pick this to Sphinx I noticed that pyflakes warns about an undefined variable in `educate_tokens()` function.

The attached one-line patch should fix it.


Attachments:

- [attr0.diff](https://sourceforge.net/p/docutils/bugs/_discuss/thread/0b5f7f4d/c469/attachment/attr0.diff) (438 Bytes; text/x-patch)


---

** [bugs:#317] Extra space inserted with French smartquotes**

**Status:** closed-fixed
**Group:** Default
**Created:** Sun Apr 23, 2017 01:17 PM UTC by Dmitry Shachnev
**Last Updated:** Mon Apr 24, 2017 09:25 AM UTC
**Owner:** nobody


Another issue [spotted by @jfbu](https://github.com/sphinx-doc/sphinx/pull/3562#issuecomment-296399918). This command:

echo '(``"entry"``)' | ./tools/rst2xml.py --smart-quotes=yes --language=fr

produces:

<paragraph>(<literal>"entry"</literal> )</paragraph>

Note the extra NBSP (U+00A0) after the `</literal>`.

Dropping either `--language=fr` or `--smart-quotes=yes` makes it disappear.

With the release version (0.13.1), U+202F is inserted instead of U+00A0.


---

Sent from sourceforge.net because docutils-***@lists.sourceforge.net is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Günter Milde]

While the attache patch may fix the pyflakes warning, I don't think it makes smartquotes work as intended. Instead I propose the following:

--- smartquotes.py (Revision 8057)
+++ smartquotes.py (Arbeitskopie)
@@ -514,9 +514,7 @@
do_ellipses = False
do_stupefy = False

- if attr == "0": # Do nothing.
- yield text
- elif attr == "1": # Do everything, turn all options on.
+ if attr == "1": # Do everything, turn all options on.
do_quotes = True
do_backticks = True
do_dashes = 1
@@ -553,13 +551,13 @@

# skip HTML and/or XML tags as well as emtpy text tokens
# without updating the last character
- if ttype == 'tag' or not text:
+ if ttype == 'tag' or attr == "0" or not text:
yield text
continue

# skip literal text (math, literal, raw, ...)
if ttype == 'literal':
- prev_token_last_char = text[-1:]
+ prev_token_last_char = text[-1:].replace('"',';').replace("'",';')
yield text
continue




---

** [bugs:#317] Extra space inserted with French smartquotes**

**Status:** closed-fixed
**Group:** Default
**Created:** Sun Apr 23, 2017 01:17 PM UTC by Dmitry Shachnev
**Last Updated:** Mon Apr 24, 2017 12:15 PM UTC
**Owner:** nobody


Another issue [spotted by @jfbu](https://github.com/sphinx-doc/sphinx/pull/3562#issuecomment-296399918). This command:

echo '(``"entry"``)' | ./tools/rst2xml.py --smart-quotes=yes --language=fr

produces:

<paragraph>(<literal>"entry"</literal> )</paragraph>

Note the extra NBSP (U+00A0) after the `</literal>`.

Dropping either `--language=fr` or `--smart-quotes=yes` makes it disappear.

With the release version (0.13.1), U+202F is inserted instead of U+00A0.


---

Sent from sourceforge.net because docutils-***@lists.sourceforge.net is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Günter Milde]

Sorry, it should also work with your patch. The simplest change, however, is to turn the first clause into a comment instead of letting it pass.


---

** [bugs:#317] Extra space inserted with French smartquotes**

**Status:** closed-fixed
**Group:** Default
**Created:** Sun Apr 23, 2017 01:17 PM UTC by Dmitry Shachnev
**Last Updated:** Mon Apr 24, 2017 03:57 PM UTC
**Owner:** nobody


Another issue [spotted by @jfbu](https://github.com/sphinx-doc/sphinx/pull/3562#issuecomment-296399918). This command:

echo '(``"entry"``)' | ./tools/rst2xml.py --smart-quotes=yes --language=fr

produces:

<paragraph>(<literal>"entry"</literal> )</paragraph>

Note the extra NBSP (U+00A0) after the `</literal>`.

Dropping either `--language=fr` or `--smart-quotes=yes` makes it disappear.

With the release version (0.13.1), U+202F is inserted instead of U+00A0.


---

Sent from sourceforge.net because docutils-***@lists.sourceforge.net is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[David Goodger]

Attached is the output of alltests.py showing a lingering issue with this feature.


Attachments:

- [alltests-cygwin.out](https://sourceforge.net/p/docutils/bugs/_discuss/thread/0b5f7f4d/dbaa/attachment/alltests-cygwin.out) (2.7 kB; application/octet-stream)


---

** [bugs:#317] Extra space inserted with French smartquotes**

**Status:** closed-fixed
**Group:** Default
**Created:** Sun Apr 23, 2017 01:17 PM UTC by Dmitry Shachnev
**Last Updated:** Mon Apr 24, 2017 04:04 PM UTC
**Owner:** nobody


Another issue [spotted by @jfbu](https://github.com/sphinx-doc/sphinx/pull/3562#issuecomment-296399918). This command:

echo '(``"entry"``)' | ./tools/rst2xml.py --smart-quotes=yes --language=fr

produces:

<paragraph>(<literal>"entry"</literal> )</paragraph>

Note the extra NBSP (U+00A0) after the `</literal>`.

Dropping either `--language=fr` or `--smart-quotes=yes` makes it disappear.

With the release version (0.13.1), U+202F is inserted instead of U+00A0.


---

Sent from sourceforge.net because docutils-***@lists.sourceforge.net is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[David Goodger]

- **status**: closed-fixed --> open



---

** [bugs:#317] Extra space inserted with French smartquotes**

**Status:** open
**Group:** Default
**Created:** Sun Apr 23, 2017 01:17 PM UTC by Dmitry Shachnev
**Last Updated:** Wed May 03, 2017 10:21 PM UTC
**Owner:** nobody


Another issue [spotted by @jfbu](https://github.com/sphinx-doc/sphinx/pull/3562#issuecomment-296399918). This command:

echo '(``"entry"``)' | ./tools/rst2xml.py --smart-quotes=yes --language=fr

produces:

<paragraph>(<literal>"entry"</literal> )</paragraph>

Note the extra NBSP (U+00A0) after the `</literal>`.

Dropping either `--language=fr` or `--smart-quotes=yes` makes it disappear.

With the release version (0.13.1), U+202F is inserted instead of U+00A0.


---

Sent from sourceforge.net because docutils-***@lists.sourceforge.net is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Günter Milde]

Thanks for pointing this out, sorry for the hazzle.
This was actually a regression in [r8062]. Fixed in [r8065].



---

** [bugs:#317] Extra space inserted with French smartquotes**

**Status:** open
**Group:** Default
**Created:** Sun Apr 23, 2017 01:17 PM UTC by Dmitry Shachnev
**Last Updated:** Wed May 03, 2017 10:21 PM UTC
**Owner:** nobody


Another issue [spotted by @jfbu](https://github.com/sphinx-doc/sphinx/pull/3562#issuecomment-296399918). This command:

echo '(``"entry"``)' | ./tools/rst2xml.py --smart-quotes=yes --language=fr

produces:

<paragraph>(<literal>"entry"</literal> )</paragraph>

Note the extra NBSP (U+00A0) after the `</literal>`.

Dropping either `--language=fr` or `--smart-quotes=yes` makes it disappear.

With the release version (0.13.1), U+202F is inserted instead of U+00A0.


---

Sent from sourceforge.net because docutils-***@lists.sourceforge.net is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Günter Milde]

- **status**: open --> closed-fixed



---

** [bugs:#317] Extra space inserted with French smartquotes**

**Status:** closed-fixed
**Group:** Default
**Created:** Sun Apr 23, 2017 01:17 PM UTC by Dmitry Shachnev
**Last Updated:** Thu May 04, 2017 01:08 PM UTC
**Owner:** nobody


Another issue [spotted by @jfbu](https://github.com/sphinx-doc/sphinx/pull/3562#issuecomment-296399918). This command:

echo '(``"entry"``)' | ./tools/rst2xml.py --smart-quotes=yes --language=fr

produces:

<paragraph>(<literal>"entry"</literal> )</paragraph>

Note the extra NBSP (U+00A0) after the `</literal>`.

Dropping either `--language=fr` or `--smart-quotes=yes` makes it disappear.

With the release version (0.13.1), U+202F is inserted instead of U+00A0.


---

Sent from sourceforge.net because docutils-***@lists.sourceforge.net is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.